In general, I like where this is going.  I agree that a stdout redirector is
probably the most common case, and for that, it almost always (for me)
redirects to an open file.  The use case for stderr redirection is usually,
but not always, to redirect stderr to stdout, but I agree that that is much
more limited.  stdin redirection is much more rare, mostly a testing device,
and better done with mocking.

A suggestion about the name.  Thinking about how it will read in a
with-statement:

    with stdout_to(some_file):
        print('hello')

Since this is all about convenience, I'd mildly suggest an API similar to
built-in open().  I'd rather write this:

    with stdout_to('/tmp/debug.log', 'w', encoding='utf-8'):
        print('hello')

than this:

    with open('/tmp/debug.log', 'w', encoding='utf-8') as tmp_stdout:
        with stdout_to(tmp_stdout):
            print('hello')

stdout_to() could optionally take a single which would be an already open file
object.

Anyway, no doubt you'll paint this bikeshed your own particular color.  Mine
only cost $0.02.