classification
Title: Python3 unbuffered stdin
Type: behavior Stage:
Components: Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Joe.Borg, pitrou, r.david.murray, sbt
Priority: normal Keywords:

Created on 2013-08-28 09:19 by Joe.Borg, last changed 2013-08-28 17:53 by pitrou. This issue is now closed.

Messages (3)
msg196359 - (view) Author: Joe Borg (Joe.Borg) Date: 2013-08-28 09:19
I'm in need of an unbuffered stdin for Python3.  Using the '-u' flag worked fine in Python2.  But, it seems, Python3's stdin is always buffered; as seen in http://bugs.python.org/issue4705.

This is not always desirable.  For example:

#!/bin/python3
import os, subprocess, time

with open("%s/unbuffered_test.log" % (os.getenv("HOME")), "w") as f:
    with subprocess.Popen(["%s/unbuffered_test.sh" % (os.getenv("HOME"))], stdin=subprocess.PIPE, stdout=f, stderr=f) as p:
        p.stdin.write(bytes("test\n", encoding="utf-8"))
        time.sleep(10)

Where unbuffered_test.sh is:
#!/bin/sh
read INPUT
echo $INPUT
exit 0

Running with -u in Python2 sees the log file populated before the 10 seconds are up.  This isn't the case in Python3.  This making controlling applications, using subprocess, basically impossible; without putting p.stdin.flush() after each command (which does work in the example above).

I ran this example in Python3.3.2.
msg196385 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-08-28 14:27
Try using Popen(..., bufsize=0).
msg196395 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-28 17:53
Indeed, this is not related to sys.stdin, but to Popen's own buffering, and the fix is either to pass "bufsize=0" or to flush() when you need to.

I'm closing as invalid, don't hesitate to re-open if I misunderstood something.
History
Date User Action Args
2013-08-28 17:53:46pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg196395

resolution: not a bug
2013-08-28 14:34:21r.david.murraysetnosy: + r.david.murray

versions: - Python 3.1, Python 3.2, Python 3.5
2013-08-28 14:27:29sbtsetnosy: + sbt
messages: + msg196385
2013-08-28 09:19:38Joe.Borgsettype: behavior
2013-08-28 09:19:24Joe.Borgcreate