classification
Title: subprocess.Popen(..., env={}) fails to pass empty env.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3, Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: Arfrever, gregory.p.smith, haypo, python-dev, r.david.murray, twouters
Priority: high Keywords: needs review, patch

Created on 2011-06-21 14:36 by twouters, last changed 2011-06-23 13:39 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
subprocess.diff.txt twouters, 2011-06-21 14:36 subprocess.diff
Messages (14)
msg138787 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2011-06-21 14:36
The addition of the _posixsubprocess module in 3.2 introduced a change of behaviour when passing an empty dict (or other false value besides None) as env:

python3.1 -c 'import subprocess; print(subprocess.Popen(["env"], env={}, stdout=subprocess.PIPE).communicate())'
(b'', None)

python3.2 -c 'import subprocess; print(subprocess.Popen(["env"], env={}, stdout=subprocess.PIPE).communicate())'
(b'LC_MONETARY=C\nSHELL=/bin/bash\nTERM=scr....', None)

The test for 'env' being the default (None) is not explicit enough.
msg138788 - (view) Author: Roundup Robot (python-dev) Date: 2011-06-21 15:24
New changeset b5963fceddad by Victor Stinner in branch '3.2':
Close #12383: Fix subprocess module with env={}: don't copy the environment
http://hg.python.org/cpython/rev/b5963fceddad

New changeset 10ecf8576eb2 by Victor Stinner in branch 'default':
(merge 3.2) Close #12383: Fix subprocess module with env={}: don't copy the
http://hg.python.org/cpython/rev/10ecf8576eb2
msg138789 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-21 15:28
> The addition of the _posixsubprocess module in 3.2 introduced
> a change of behaviour when passing an empty dict
> (or other false value besides None) as env: ...

This bug was introduced by the commit (768722b2ae0a) introducing _posixsubprocess.
msg138790 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-21 15:28
Thanks for the fix, I added an unit test.
msg138791 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-21 16:32
The test appears to be failing on the buildbots:

http://www.python.org/dev/buildbot/all/builders/x86%20Ubuntu%20Shared%203.2/builds/370
msg138792 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2011-06-21 18:21
That failure is likely due to the environment being empty causing the subprocess to be unable to run.  Notice this error message further up:

test_empty_env (test.test_subprocess.ContextManagerTests) ... /srv/buildbot/buildarea/3.2.bolen-ubuntu/build/python: error while loading shared libraries: libpython3.2dm.so.1.0: cannot open shared object file: No such file or directory
FAIL

Looking at the environment at the top of the buildbot run log I'm not sure what it would be.
msg138794 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-21 19:52
On Mac OS X, the failure is different:

======================================================================
FAIL: test_empty_env (test.test_subprocess.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_subprocess.py", line 400, in test_empty_env
    self.assertEqual(stdout.strip(), b"0")
AssertionError: b'1' != b'0'

http://www.python.org/dev/buildbot/all/builders/PPC%20Tiger%203.x/builds/2059/steps/test/logs/stdio
msg138796 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-06-21 19:58
Maybe test_empty_env() should pass LD_LIBRARY_PATH to child process.
msg138797 - (view) Author: Roundup Robot (python-dev) Date: 2011-06-21 19:59
New changeset 8d46e84cd812 by Victor Stinner in branch '3.2':
Issue #12383: skip test_empty_env() if compiled is compiled in shared mode
http://hg.python.org/cpython/rev/8d46e84cd812
msg138798 - (view) Author: Roundup Robot (python-dev) Date: 2011-06-21 20:00
New changeset d1d5a7392e39 by Victor Stinner in branch 'default':
(merge 3.2) Issue #12383: skip test_empty_env() if compiled is compiled in
http://hg.python.org/cpython/rev/d1d5a7392e39
msg138800 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-21 20:01
> Maybe test_empty_env() should pass LD_LIBRARY_PATH to child process.

The idea of the test is to test an empty environment: if we pass one variable, it is no more an empty environment.

I changed the test to skip it if Python is compiled in shared mode.
msg138831 - (view) Author: Roundup Robot (python-dev) Date: 2011-06-22 19:29
New changeset da3af4b131d7 by Victor Stinner in branch '3.2':
Issue #12383: fix test_empty_env() of subprocess on Mac OS X
http://hg.python.org/cpython/rev/da3af4b131d7

New changeset 29819072855a by Victor Stinner in branch 'default':
(merge 3.2) Issue #12383: fix test_empty_env() of subprocess on Mac OS X
http://hg.python.org/cpython/rev/29819072855a
msg138837 - (view) Author: Roundup Robot (python-dev) Date: 2011-06-22 23:04
New changeset 52c5f80122dd by Victor Stinner in branch '3.2':
Issue #12383: skip test_empty_env() of subprocess on Windows
http://hg.python.org/cpython/rev/52c5f80122dd

New changeset 93cd98782f47 by Victor Stinner in branch 'default':
(merge 3.2) Issue #12383: skip test_empty_env() of subprocess on Windows
http://hg.python.org/cpython/rev/93cd98782f47
msg138867 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-23 13:39
All Python 3.x buildbots are green again (except FreeBSD 7.2, but the failures are not related to this issue).
History
Date User Action Args
2011-06-23 13:39:49hayposetstatus: open -> closed

messages: + msg138867
2011-06-22 23:04:37python-devsetmessages: + msg138837
2011-06-22 19:29:10python-devsetmessages: + msg138831
2011-06-21 20:01:37hayposetmessages: + msg138800
2011-06-21 20:00:03python-devsetmessages: + msg138798
2011-06-21 19:59:03python-devsetmessages: + msg138797
2011-06-21 19:58:14Arfreversetnosy: + Arfrever
messages: + msg138796
2011-06-21 19:52:29hayposetmessages: + msg138794
2011-06-21 18:21:58gregory.p.smithsetmessages: + msg138792
2011-06-21 16:32:05r.david.murraysetstatus: closed -> open
nosy: + r.david.murray
messages: + msg138791

2011-06-21 15:28:26hayposetmessages: + msg138790
2011-06-21 15:28:07hayposetnosy: + haypo
messages: + msg138789
2011-06-21 15:24:18python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg138788

resolution: fixed
stage: patch review -> resolved
2011-06-21 14:36:01twouterscreate