classification
Title: Support bytes for subprocess.Popen()
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.1
process
Status: closed Resolution: fixed
Dependencies: 4035 Superseder:
Assigned To: Nosy List: a.badger, brian.curtin, loewis, vstinner
Priority: high Keywords: patch

Created on 2008-10-03 23:45 by vstinner, last changed 2010-04-20 23:51 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
subprocess-bytes.patch vstinner, 2008-10-08 00:43 Support bytes in the POSIX version of subprocess
subprocess-bytes-2.patch vstinner, 2010-03-03 22:37
Messages (12)
msg74284 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2008-10-03 23:45
subprocess doesn't support bytes for the "args" argument.
 - On Windows, subprocess._execute_child() converts args to a string 
if it was a list
 - On UNIX, subprocess._execute_child() converts args to a list if 
it's a string

If shell=True, _execute_child() adds a prefix to the arguments.

I don't know if subprocess should accept bytes in a command line. 
Attached patch fixes POSIX version of subprocess to support bytes.

See also related issue #4035.
msg74305 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-04 08:16
I think this should be deferred to 3.1. I'm not sure how much
consistency across platforms is desirable, however, using strings is
surely the better choice on Windows. I could sympathize with providing
bytes support only on POSIX, but I'm sure others will disagree.
msg74505 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2008-10-08 00:43
Oops, I attached the wrong patch :-/
msg75366 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2008-10-30 09:52
@loewis: My patch only changes the behaviour of the POSIX version of 
subprocess, Windows version is unchanged.
msg100357 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-03-03 22:37
os.exec*() now accept bytes thanks to the PEP 383: see issue #4035.

I updated my patch: it now includes tests \o/ It works on Linux. Can someone test it on Windows and/or Mac OS X?
msg100360 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-03-04 00:25
(oops, the issue is still open)
msg100363 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-03-04 00:53
The tests pass on OS X, but fail on Windows. I'll look into it.
msg100365 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-03-04 01:12
Oops, I realized that the second test is just useless. The argument is str, not bytes. I wanted to test Popen(bytes, shell=True). What is the right encoding to convert a string to bytes for the file system?
msg100366 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-03-04 01:26
The list2cmdline function checks spaces, tabs, etc in line 521 of subprocess.py. In your first test case, it ends up checking if a string is contained in a bytes object, which is a TypeError for the str not supporting the buffer API. 

Would it be acceptable to just add "arg = str(arg)" right under the for loop? That fixes it for me, not sure of the preferred way to go about it.
msg100367 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-03-04 01:30
> What is the right encoding to convert a string to bytes 
> for the file system?

sys.getfilesystemencoding()?
msg100376 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-03-04 06:43
On Windows, command lines shouldn't need to be encoded in any encoding. Instead, the unicode string should be passed to the system call as-is.
msg103796 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-04-20 23:51
subprocess accepts byte string in program arguments in py3k, so the issue can be closed.

I opened other issues for the other arguments:
 - current working directory: #8393
 - environment variables: #8391
 - error message: #8467
History
Date User Action Args
2010-04-20 23:51:31vstinnersetstatus: open -> closed
keywords: patch, patch
resolution: fixed
2010-04-20 23:51:14vstinnersetkeywords: patch, patch

messages: + msg103796
2010-03-04 06:43:51loewissetkeywords: patch, patch

messages: + msg100376
2010-03-04 01:30:12brian.curtinsetkeywords: patch, patch

messages: + msg100367
2010-03-04 01:26:33brian.curtinsetkeywords: patch, patch

messages: + msg100366
2010-03-04 01:12:55vstinnersetkeywords: patch, patch

messages: + msg100365
2010-03-04 00:53:30brian.curtinsetkeywords: patch, patch
nosy: + brian.curtin
messages: + msg100363

2010-03-04 00:25:09vstinnersetstatus: closed -> open
keywords: patch, patch
resolution: fixed -> (no value)
messages: + msg100360
2010-03-03 22:37:11vstinnersetstatus: open -> closed
files: + subprocess-bytes-2.patch
messages: + msg100357

keywords: patch, patch
resolution: fixed
2009-05-17 02:43:10ajaksu2setpriority: high
keywords: patch, patch
type: enhancement
components: + Library (Lib)
versions: + Python 3.1, - Python 3.0
2008-10-30 09:52:18vstinnersetkeywords: patch, patch
messages: + msg75366
2008-10-15 15:22:15a.badgersetnosy: + a.badger
2008-10-08 00:43:39vstinnersetkeywords: patch, patch
files: + subprocess-bytes.patch
dependencies: + Support bytes for os.exec*()
messages: + msg74505
2008-10-08 00:42:47vstinnersetfiles: - os_exec_bytes.patch
2008-10-04 08:16:45loewissetkeywords: patch, patch
nosy: + loewis
messages: + msg74305
2008-10-03 23:45:14vstinnercreate