http://docs.python.org/3/library/subprocess.html

A common confusion is that result from subprocess calls in Python 3 is bytes, not string, and needs to be converted.

The problem is complicated because you need to know the encoding of input/output streams. This should be documented at least.

http://stackoverflow.com/questions/606191/convert-byte-array-to-python-string

Actually stdin/stdout/stderr are string streams if universal_newline is True

>
>
> Actually stdin/stdout/stderr are string streams if universal_newline is
> True
>

I believe that makes the issue even worse. =)

Anatoly, do you have a specific suggestion for improved wording?  This *is* documented in the subprocess documentation, in several appropriate places, including the fact that the appropriate encoding to use may need to be determined at the application level.

The attached patch explicitly mentions that stdin/stdout/stderr are opened as binary streams when universal_newlines is False. 

I'm not convinced that adding this text will solve the confusion, subprocess has fairly complex documentation due to the rich API and it is easy to mis information when you don't read carefully. That said, the documentation for check_output (which is likely the most common way to run a command and fetch its output) is already clear about the types of the returned data, both in prose and example code.

The only thing that can save the docs is pictures. Or tables.

          +---------+---------------------------+
          |         | universal_newlines = True |
   +------+---------+---------------------------+
   |Py.2  |   str   |         str               |
   +------+         |                           |
   |Py.3  |  bytes  |         str               |
   +------+---------+---------------------------+

I don't agree that adding a table/picture is the only thing that can be done, or even that it is a good idea.

IMHO the subprocess documentation is clear enough about the distinction between bytes and string, especially in the section about convenience functions that is right at the top of the page.

Also in the Python 3 docs we don't compare the current behavior with Python 2.  Ronald patch LGTM.

LGTM.

On Thu, Jun 13, 2013 at 9:20 AM, Ezio Melotti <report@bugs.python.org>wrote:

>
> Also in the Python 3 docs we don't compare the current behavior with
> Python 2.
>

That's most unfortunate. Major PITA comes from attempts to port existing
code.

_failed_ attempts to port existing code.

Surely there are already good places to help with 2->3 transition?
The Library Reference is not such a place.

New changeset a2c2ffa1a41c by Ronald Oussoren in branch '3.3':
Issue #17860: explicitly mention that std* streams are opened in binary mode by default.
http://hg.python.org/cpython/rev/a2c2ffa1a41c

New changeset ae8b054155c1 by Ronald Oussoren in branch 'default':
(3.3->default) Issue #17860: explicitly mention that std* streams are opened in binary mode by default.
http://hg.python.org/cpython/rev/ae8b054155c1

The documentation for 3.3 and default now explicitly mentions that the std* streams are opened as binary streams when universal_newlines is false.

New changeset f4747e1ce2b1 by Ronald Oussoren in branch '3.3':
Cleanup of documentation change from #17860
http://hg.python.org/cpython/rev/f4747e1ce2b1

New changeset 83810f67d16b by Ronald Oussoren in branch 'default':
(3.3->default) Cleanup of documentation change from #17860
http://hg.python.org/cpython/rev/83810f67d16b