This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: subprocess.Popen(..., env={}) fails to pass empty env.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: Arfrever, gregory.p.smith, ned.deily, python-dev, r.david.murray, twouters, vstinner
Priority: high Keywords: needs review, patch

Created on 2011-06-21 14:36 by twouters, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
subprocess.diff.txt twouters, 2011-06-21 14:36 subprocess.diff
Pull Requests
URL Status Linked Edit
PR 1874 merged gregory.p.smith, 2017-05-30 21:26
PR 1877 merged gregory.p.smith, 2017-05-30 23:15
PR 3278 merged ned.deily, 2017-09-04 01:01
Messages (19)
msg138787 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2011-06-21 14:36
The addition of the _posixsubprocess module in 3.2 introduced a change of behaviour when passing an empty dict (or other false value besides None) as env:

python3.1 -c 'import subprocess; print(subprocess.Popen(["env"], env={}, stdout=subprocess.PIPE).communicate())'
(b'', None)

python3.2 -c 'import subprocess; print(subprocess.Popen(["env"], env={}, stdout=subprocess.PIPE).communicate())'
(b'LC_MONETARY=C\nSHELL=/bin/bash\nTERM=scr....', None)

The test for 'env' being the default (None) is not explicit enough.
msg138788 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-21 15:24
New changeset b5963fceddad by Victor Stinner in branch '3.2':
Close #12383: Fix subprocess module with env={}: don't copy the environment
http://hg.python.org/cpython/rev/b5963fceddad

New changeset 10ecf8576eb2 by Victor Stinner in branch 'default':
(merge 3.2) Close #12383: Fix subprocess module with env={}: don't copy the
http://hg.python.org/cpython/rev/10ecf8576eb2
msg138789 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-06-21 15:28
> The addition of the _posixsubprocess module in 3.2 introduced
> a change of behaviour when passing an empty dict
> (or other false value besides None) as env: ...

This bug was introduced by the commit (768722b2ae0a) introducing _posixsubprocess.
msg138790 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-06-21 15:28
Thanks for the fix, I added an unit test.
msg138791 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-21 16:32
The test appears to be failing on the buildbots:

http://www.python.org/dev/buildbot/all/builders/x86%20Ubuntu%20Shared%203.2/builds/370
msg138792 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2011-06-21 18:21
That failure is likely due to the environment being empty causing the subprocess to be unable to run.  Notice this error message further up:

test_empty_env (test.test_subprocess.ContextManagerTests) ... /srv/buildbot/buildarea/3.2.bolen-ubuntu/build/python: error while loading shared libraries: libpython3.2dm.so.1.0: cannot open shared object file: No such file or directory
FAIL

Looking at the environment at the top of the buildbot run log I'm not sure what it would be.
msg138794 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-06-21 19:52
On Mac OS X, the failure is different:

======================================================================
FAIL: test_empty_env (test.test_subprocess.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_subprocess.py", line 400, in test_empty_env
    self.assertEqual(stdout.strip(), b"0")
AssertionError: b'1' != b'0'

http://www.python.org/dev/buildbot/all/builders/PPC%20Tiger%203.x/builds/2059/steps/test/logs/stdio
msg138796 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2011-06-21 19:58
Maybe test_empty_env() should pass LD_LIBRARY_PATH to child process.
msg138797 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-21 19:59
New changeset 8d46e84cd812 by Victor Stinner in branch '3.2':
Issue #12383: skip test_empty_env() if compiled is compiled in shared mode
http://hg.python.org/cpython/rev/8d46e84cd812
msg138798 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-21 20:00
New changeset d1d5a7392e39 by Victor Stinner in branch 'default':
(merge 3.2) Issue #12383: skip test_empty_env() if compiled is compiled in
http://hg.python.org/cpython/rev/d1d5a7392e39
msg138800 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-06-21 20:01
> Maybe test_empty_env() should pass LD_LIBRARY_PATH to child process.

The idea of the test is to test an empty environment: if we pass one variable, it is no more an empty environment.

I changed the test to skip it if Python is compiled in shared mode.
msg138831 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-22 19:29
New changeset da3af4b131d7 by Victor Stinner in branch '3.2':
Issue #12383: fix test_empty_env() of subprocess on Mac OS X
http://hg.python.org/cpython/rev/da3af4b131d7

New changeset 29819072855a by Victor Stinner in branch 'default':
(merge 3.2) Issue #12383: fix test_empty_env() of subprocess on Mac OS X
http://hg.python.org/cpython/rev/29819072855a
msg138837 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-22 23:04
New changeset 52c5f80122dd by Victor Stinner in branch '3.2':
Issue #12383: skip test_empty_env() of subprocess on Windows
http://hg.python.org/cpython/rev/52c5f80122dd

New changeset 93cd98782f47 by Victor Stinner in branch 'default':
(merge 3.2) Issue #12383: skip test_empty_env() of subprocess on Windows
http://hg.python.org/cpython/rev/93cd98782f47
msg138867 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-06-23 13:39
All Python 3.x buildbots are green again (except FreeBSD 7.2, but the failures are not related to this issue).
msg294789 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-05-30 21:40
New changeset b351248c1fa74277b886b0c9a2e1c2f65fabe9ef by Gregory P. Smith in branch 'master':
bpo-12383: Refactor subprocess test_empty_env (#1874)
https://github.com/python/cpython/commit/b351248c1fa74277b886b0c9a2e1c2f65fabe9ef
msg294793 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-05-30 22:32
Greg, b351248c1fa74277b886b0c9a2e1c2f65fabe9ef appears to have broken the macOS buildbots:
http://buildbot.python.org/all/builders/x86-64%20Sierra%203.x/builds/233
http://buildbot.python.org/all/builders/x86-64%20El%20Capitan%203.x/builds/235
msg294797 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-05-30 22:51
i wondered if that might happen.  the test skipping logic that wasn't correct was added for macos; probably needs to be written another way.  i've got a mac today, i'll figure it out.
msg294799 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-05-30 23:34
fixed in https://github.com/python/cpython/commit/85aba238e49abd2d5a604102981d28a50f305443
msg301205 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-09-04 04:00
New changeset 918edc0edb356d0561062c1dc267b1d68a684b70 by Ned Deily in branch 'master':
bpo-12383: Also ignore __PYVENV_LAUNCHER__ (#3278)
https://github.com/python/cpython/commit/918edc0edb356d0561062c1dc267b1d68a684b70
History
Date User Action Args
2022-04-11 14:57:18adminsetgithub: 56592
2017-09-04 04:00:24ned.deilysetmessages: + msg301205
2017-09-04 01:01:11ned.deilysetpull_requests: + pull_request3321
2017-05-30 23:34:55gregory.p.smithsetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2017-05-30 23:34:37gregory.p.smithsetmessages: + msg294799
2017-05-30 23:15:14gregory.p.smithsetpull_requests: + pull_request1957
2017-05-30 22:51:44gregory.p.smithsetmessages: + msg294797
2017-05-30 22:32:45ned.deilysetstatus: closed -> open

nosy: + ned.deily
messages: + msg294793

resolution: fixed -> (no value)
stage: resolved -> needs patch
2017-05-30 21:40:40gregory.p.smithsetmessages: + msg294789
2017-05-30 21:26:36gregory.p.smithsetpull_requests: + pull_request1954
2011-06-23 13:39:49vstinnersetstatus: open -> closed

messages: + msg138867
2011-06-22 23:04:37python-devsetmessages: + msg138837
2011-06-22 19:29:10python-devsetmessages: + msg138831
2011-06-21 20:01:37vstinnersetmessages: + msg138800
2011-06-21 20:00:03python-devsetmessages: + msg138798
2011-06-21 19:59:03python-devsetmessages: + msg138797
2011-06-21 19:58:14Arfreversetnosy: + Arfrever
messages: + msg138796
2011-06-21 19:52:29vstinnersetmessages: + msg138794
2011-06-21 18:21:58gregory.p.smithsetmessages: + msg138792
2011-06-21 16:32:05r.david.murraysetstatus: closed -> open
nosy: + r.david.murray
messages: + msg138791

2011-06-21 15:28:26vstinnersetmessages: + msg138790
2011-06-21 15:28:07vstinnersetnosy: + vstinner
messages: + msg138789
2011-06-21 15:24:18python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg138788

resolution: fixed
stage: patch review -> resolved
2011-06-21 14:36:01twouterscreate