I think this should also be added to the whatsnew.

Regarding the examples, isn't it easier to say that:
  with redirect_stdout(sys.stderr):
      print('error')
is equivalent to
  print('error', file=sys.stderr)
?

I think that in most of the cases users are redirecting something that is being print()ed, and this example gets the point across (even if the "file" arg can be used for this specific case, it is not always the case if print() is called by a function).  Capturing help() and especially did.dis() output don't seem to me realistic/intuitive use cases for redirect_stdout().