classification
Title: subprocess.Popen.communicate with universal_newlines=True doesn't accept strings on 3.2
Type: behavior Stage: resolved
Components: IO, Library (Lib), Unicode Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: asvetlov, chris.jerdonek, ezio.melotti, haypo, jkloth, python-dev, serhiy.storchaka
Priority: critical Keywords: patch

Created on 2013-01-09 09:28 by serhiy.storchaka, last changed 2013-02-04 14:59 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
subprocess_communicate_string-3.2.patch serhiy.storchaka, 2013-01-26 21:05 review
subprocess_communicate_string-3.2_2.patch serhiy.storchaka, 2013-01-30 14:06 review
Messages (6)
msg179425 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-09 09:28
On 3.2 subprocess.Popen.communicate with universal_newlines=True accepts bytes and doesn't accept strings. 

$ ./python -c "import subprocess; subprocess.Popen(['cat'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True).communicate('qwerty')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/serhiy/py/cpython3.2/Lib/subprocess.py", line 833, in communicate
    return self._communicate(input)
  File "/home/serhiy/py/cpython3.2/Lib/subprocess.py", line 1470, in _communicate
    stdout, stderr = self._communicate_with_poll(input)
  File "/home/serhiy/py/cpython3.2/Lib/subprocess.py", line 1537, in _communicate_with_poll
    input_offset += os.write(fd, chunk)
TypeError: 'str' does not support the buffer interface

On 3.3+ it accepts strings and doesn't accept bytes.
msg179427 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-09 09:33
As quick fix I propose add support of strings (in additional to bytes) in 3.2. Perhaps we must also add support of bytes with warning in 3.3.
msg180712 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-26 21:05
Here is a patch which adds support of strings in communicate(). It also contains a backported test from changeset 4af2c0687970 which tests this behavior.
msg180974 - (view) Author: Jeremy Kloth (jkloth) * Date: 2013-01-30 11:03
I would suggest that this be bumped to release blocker as currently passing bytes to communicate with universal_newlines=True on Windows causes the process to hang (see issue17077).

The backported test case passes on Windows without any changes to subprocess.py so this is a POSIX-only bug.
msg180975 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-30 14:06
Indeed, Popen.communicate() is incompatible not only between 3.2 and 3.3, but between Windows and POSIX too.

Documentation says the input argument should be a bytes string and 3.2 on POSIX works only with bytes strings. 3.2 on Windows and 3.3 on both platforms require an unicode string, as 3.3 documentation says. 3.2 on Windows hangs with a bytes string.

Patch updated with the documentation change (copied from 3.3).
msg181330 - (view) Author: Roundup Robot (python-dev) Date: 2013-02-04 14:52
New changeset 4206f91c974c by Serhiy Storchaka in branch '3.2':
Issue #16903: Popen.communicate() on Unix now accepts strings when
http://hg.python.org/cpython/rev/4206f91c974c
History
Date User Action Args
2013-02-04 14:59:50serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-02-04 14:52:14python-devsetnosy: + python-dev
messages: + msg181330
2013-01-30 14:06:58serhiy.storchakasetpriority: normal -> critical
files: + subprocess_communicate_string-3.2_2.patch
messages: + msg180975
2013-01-30 13:31:46serhiy.storchakalinkissue17077 dependencies
2013-01-30 12:28:28hayposetnosy: + haypo
2013-01-30 11:03:12jklothsetnosy: + jkloth
messages: + msg180974
2013-01-26 21:05:05serhiy.storchakasetfiles: + subprocess_communicate_string-3.2.patch

assignee: serhiy.storchaka

keywords: + patch
nosy: + asvetlov
messages: + msg180712
stage: needs patch -> patch review
2013-01-14 11:09:17serhiy.storchakasetstage: needs patch
2013-01-09 10:10:47chris.jerdoneksetnosy: + chris.jerdonek
2013-01-09 09:33:24serhiy.storchakasetmessages: + msg179427
2013-01-09 09:28:57serhiy.storchakacreate