This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: subprocess line-buffering only works in universal newlines mode
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, akira, benjamin.peterson, gregory.p.smith, hynek, pitrou, stutzbach
Priority: normal Keywords:

Created on 2014-05-11 11:25 by pitrou, last changed 2022-04-11 14:58 by admin.

Messages (4)
msg218257 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-05-11 11:25
The docs for subprocess.Popen seem to imply that line-buffering is always available. However, bufsize=1 is a special value only when open the pipes in text mode, i.e. when "universal newlines" are enabled.

In the short term, we should probably fix the subprocess docs.
In the long term, perhaps we can add a line buffering capability to BufferedWriter?
msg218259 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-05-11 11:35
Ha, it seems actually worse than that, since no buffering argument is ever passed to the TextIOWrapper constructor. "bufsize=1" will simply get ignored, and line buffering doesn't work at all.

Example under 2.7:

$ python -c 'import subprocess; p = subprocess.Popen(["/bin/cat"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True); p.stdin.write("foo\n"); print(p.stdout.readline()); p.stdin.close()'
foo

Under 3.4, the same line hangs in the p.stdout.readline() call.
msg218260 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-05-11 11:39
Ok, the "not working at all" part is issue #21332. Let's make this issue specific to binary mode, again.
msg218263 - (view) Author: Akira Li (akira) * Date: 2014-05-11 12:20
Until the current patch for issue #21332 is committed; bufsize=1 is 
equivalent to bufsize=-1 in both binary and text mode. 

You are correct the patch in #21332 fixes only the text mode
(universal_newlines=True) -- it also updates the documentation
to mention that bufsize=1 works only in text mode.
History
Date User Action Args
2022-04-11 14:58:03adminsetgithub: 65670
2014-05-11 12:20:06akirasetmessages: + msg218263
2014-05-11 11:39:16pitrousetnosy: + akira

messages: + msg218260
title: subprocess line-buffering doesn't work -> subprocess line-buffering only works in universal newlines mode
2014-05-11 11:35:53pitrousetmessages: + msg218259
title: subprocess line-buffering only works in universal newlines mode -> subprocess line-buffering doesn't work
2014-05-11 11:33:19Arfreversetnosy: + Arfrever
2014-05-11 11:25:06pitroucreate