classification
Title: subprocess: canonicalize env to bytes on Unix (Python3)
Type: Stage:
Components: Library (Lib), Unicode Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: 8513 Superseder:
Assigned To: Nosy List: Arfrever, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2010-05-06 22:45 by vstinner, last changed 2010-05-18 20:41 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
subprocess_canonalize_env.patch vstinner, 2010-05-16 02:22
Messages (7)
msg105168 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-06 22:45
Python 3.2 has now its os.environb, the bytes version of os.environ. subprocess should get a new envb argument to be able to use pure bytes environmental variables. Examples:

   subprocess.call([b'env], envb={b'PATH': b'/usr/bin'})

and

   envb = os.environb.copy()
   envb[b'PATH'] = b'/usr/bin'
   subprocess.call([b'env], envb=envb)

Specify both env and envb would raise an exception. envb should only be available on POSIX (as os.environb).

Related issues: #8513 (subprocess: support bytes program name) and #8603 (os.environb).
msg105174 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-05-06 23:48
Why wouldn't you give byte variables in env too?
msg105696 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-14 10:28
> Why wouldn't you give byte variables in env too?

The problem with the canonicalization to bytes is to choice of the preferred type. Eg. env={'PATH': 'a', b'PATH': b'b'}: should we use 'a', 'b' or raise an error?

subprocess does already convert environ keys and values to bytes on Unix (in subprocess._execute_child() if _posixsubprocess module is present, and in posix_execve()).

os.get_exec_path() should also support b'PATH' key.
msg105844 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-16 02:22
pitou> Why wouldn't you give byte variables in env too?

Ok, attached patch canonicalize env keys and values to bytes. If a variable is defined twice (str name, bytes name), a ValueError is raised.

The patch depends on #8513: it requires that os.get_exec_path() is patched to support b'PATH' key.
msg105845 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-16 02:29
Python 3.1 accepts duplicate variables (str name, bytes name). It creates the two variables is a random order:

>>> subprocess.call(['env'], env={'xx': 'str', b'xx': 'bytes'})
xx=str
xx=bytes
0
>>> subprocess.call(['env'], env={'xxx': 'str', b'xxx': 'bytes'})
xxx=bytes
xxx=str
0
msg105961 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-18 10:35
os.exeve() and os.exevpe() should also canonicalize env to bytes.

os.exeve() and os.exevpe(), but not os._exevpe() to avoid doing it twice (once in subprocess, once in os._exevpe). Patch os._exevpe() is not enough because subprocess doesn't call it on Unix if _subprocessposix module is present.
msg106006 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-18 20:41
My patch to fix #8513 does also fix the examples of this issue and so I think that the canonicalization is no more needed. I will only reopen the issue if I find a good reason to apply the patch :-)
History
Date User Action Args
2010-05-18 20:41:52vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg106006
2010-05-18 10:35:51vstinnersetmessages: + msg105961
2010-05-16 02:29:34vstinnersetmessages: + msg105845
2010-05-16 02:22:48vstinnersetfiles: + subprocess_canonalize_env.patch
title: subprocess: add envb argument to Popen constructor (Python3, POSIX only) -> subprocess: canonicalize env to bytes on Unix (Python3)
messages: + msg105844

dependencies: + subprocess: support bytes program name (POSIX)
keywords: + patch
2010-05-14 10:28:28vstinnersetmessages: + msg105696
2010-05-06 23:48:29pitrousetnosy: + pitrou
messages: + msg105174
2010-05-06 23:00:29Arfreversetnosy: + Arfrever
2010-05-06 22:45:15vstinnercreate