classification
Title: sys.stdout.write does not allow bytes in Python 3.x
Type: behavior Stage:
Components: Documentation Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, ezio.melotti, loewis, ntrrgc, r.david.murray, sbt
Priority: normal Keywords:

Created on 2013-07-20 02:31 by ntrrgc, last changed 2013-10-13 17:56 by georg.brandl. This issue is now closed.

Messages (7)
msg193394 - (view) Author: Alicia Boya García (ntrrgc) Date: 2013-07-20 02:31
Sometimes developers need to write text to stdout, and it's nice to have on the fly Unicode to UTF-8 conversion (or what matches the platform), but sometimes they also need to output binary blobs, like text encoded in other codifications than the system default, binary files, etc.

Python2 does the thing more-or-less right and allows writing both text and binary. I think Python3 should also accept both.
msg193404 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-07-20 10:50
You can do

    sys.stdout.buffer.write(b"hello")

See

    http://docs.python.org/dev/library/io.html?highlight=buffer#io.TextIOBase.buffer
msg193407 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-20 13:07
The fundamental behavior here is not going to change, as maintaining a bright line between bytes and strings is central to Python3's design.

The question is, does the method for achieving your goal need better documentation?
msg193604 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-07-23 14:57
Rephrasing R.David's question: 

ntrrgc: Did you consult any documentation (browsing or searching), and if so, where did you look?
msg194808 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-08-10 15:51
> You can do
>    sys.stdout.buffer.write(b"hello")

Note that this might not work when sys.stdout has been replaced with something else (e.g. a StringIO).
msg195593 - (view) Author: Alicia Boya García (ntrrgc) Date: 2013-08-19 01:54
Sorry for the late response, GMail's SPAM filter ate the replies.

The main issue is sys.stdout being opened as text instead of binary. This fact is stated in the docs. http://docs.python.org/3/library/sys.html#sys.stdout

In any case, there are some caveats worth noting:

> You can do
>    sys.stdout.buffer.write(b"hello")
This is problematic if both buffer and IOTextWrapper are used. For example:

  print("Hello", end=""); sys.stdout.buffer.write(b"World")

That line may write `WorldHello` instead of `HelloWorld` (and it does indeed, at least in Linux implementation).

Yes, an application should not do this in Python3, but using print() and writing to stdout were OK in Python2, which makes porting programs harder.

A workaround is to perform sys.stdout.flush() before sys.stdout.buffer.write().

> (from the docs)
> Using io.TextIOBase.detach(), streams can be made binary by default.
> sys.stdout = sys.stdout.detach()

This should help in cases where most output is binary, but it's worth noting that interactive shells (such as the builtin python or IPython) and debuggers (both pdb and ipdb) stop working when this is used. Also, it will probably break every function there that relies on sys.stdout being Unicode or binary depending on only the Python version.
msg195699 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-08-20 15:21
> A workaround is to perform sys.stdout.flush() before 
> sys.stdout.buffer.write().

FWIW you can just use print(..., flush=True) from 3.3+.
History
Date User Action Args
2013-10-13 17:56:31georg.brandlsetstatus: pending -> closed
resolution: wont fix
2013-10-01 20:26:16serhiy.storchakasetstatus: open -> pending
2013-08-20 15:21:26ezio.melottisetmessages: + msg195699
2013-08-19 01:54:34ntrrgcsetmessages: + msg195593
2013-08-10 15:51:47ezio.melottisetnosy: + ezio.melotti
messages: + msg194808
2013-07-23 14:57:32loewissetnosy: + loewis
messages: + msg193604
2013-07-20 13:07:59r.david.murraysetversions: + Python 3.4
nosy: + docs@python, r.david.murray

messages: + msg193407

assignee: docs@python
components: + Documentation, - Library (Lib)
2013-07-20 10:50:23sbtsetnosy: + sbt
messages: + msg193404
2013-07-20 02:31:21ntrrgccreate