This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: environment inspection and manipulation API is buggy, inconsistent with "Python philosophy" for wrapping native APIs
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: os.getenv() not updated after external module uses C putenv()
View: 1159
Assigned To: Nosy List: BreamoreBoy, amaury.forgeotdarc, eric.araujo, exarkun, martin.panter
Priority: normal Keywords:

Created on 2009-01-08 22:08 by exarkun, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (6)
msg79448 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2009-01-08 22:08
According to a recent thread on python-dev, Python APIs with the same
name as native APIs can be trusted to just pass through to the platform
API call
(<http://mail.python.org/pipermail/python-dev/2009-January/084899.html>). 
This isn't the case for the environment APIs in the os module.  For
example, os.getenv doesn't always return the actual value of a key in
the environment.  Instead, it looks in the os.environ pseudo-dict to get
the answer.  This can be wrong because os.environ is a *copy* of the
environment which any non-Python API will use, and as a copy it can get
out of sync with the real environment.  One way it can get out of sync
is via os.environ.pop (at least on POSIX - os.environ has different
implementations on different platforms, I find the way an implementation
is selected to be rather confusing and I have not tracked down the
precise behavior for all platforms):

>>> import os
>>> os.environ.pop('PATH')
'/home/exarkun/.local/sbin:/home/exarkun/.local/bin:/home/exarkun/Projects/combinator_paths/bincache:/home/exarkun/.local/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games'
>>> os.getenv('PATH')
>>> os.system("echo $PATH")
/home/exarkun/.local/sbin:/home/exarkun/.local/bin:/home/exarkun/Projects/combinator_paths/bincache:/home/exarkun/.local/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games
0
>>> 

Another way is via calls to the platform environment manipulation APIs
(setenv, unsetenv or putenv).  These might be invoked by a third-party
library which is exposed to Python or via ctypes.

`environ´ itself is a C API, though it's a char**, whereas os.environ is
a Python dict.  It's probably convenient for os.environ to remain as a
dict, since that's so much simpler to manipulate and inspect than the C
data structure used.  However, it'd be good if were always in sync with
the real process environment.  A good way to do this would probably be
to stop maintaining a copy of the environment and always pass through to
one of the platform APIs to satisfy a method call onto it.
msg220765 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-16 21:09
Is the OP interested in taking this forward?
msg220815 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2014-06-17 12:32
What are the chances a future Python 2.x release will include any fix developed for this issue?
msg220816 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-17 12:39
Good as 2.7 is in support until 2020.
msg227247 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-09-22 02:12
I suspect the $PATH example is a bad example. On my Linux setup, my initial PATH value is a customized path. After popping it and calling system(), I see a simpler default path, which is probably re-initialized somewhere else.

I suspect there is no environ.pop() bug. Everthing works according the documentation if you use another environment variable:

$ TEST_VARIABLE=value python
Python 3.4.0 (default, Mar 17 2014, 23:20:09) 
[GCC 4.8.2 20140206 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('echo "$TEST_VARIABLE"')
value
0
>>> os.environ.pop("TEST_VARIABLE")
'value'
>>> os.system('echo "$TEST_VARIABLE"')

0

The other point in this report, about synchronizing “os.environ” with the C APIs, is a duplicate of Issue 1159.
msg231709 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2014-11-26 13:41
Agreed. environ.pop() was fixed a long time ago with issue1287.
It seems that all mutable methods of the environ pseudo-dict are now correctly reflected to the platform environ.

The other direction (updates from C code should be reflected in os.environ) is tracked by issue1159.
History
Date User Action Args
2022-04-11 14:56:43adminsetgithub: 49137
2014-11-26 13:41:51amaury.forgeotdarcsetstatus: open -> closed

nosy: + amaury.forgeotdarc
messages: + msg231709

superseder: os.getenv() not updated after external module uses C putenv()
resolution: duplicate
2014-09-22 02:12:49martin.pantersetmessages: + msg227247
2014-06-17 21:32:36martin.pantersetnosy: + martin.panter
2014-06-17 12:39:22BreamoreBoysetmessages: + msg220816
versions: + Python 2.7, Python 3.5
2014-06-17 12:32:19exarkunsetmessages: + msg220815
2014-06-16 21:09:26BreamoreBoysetnosy: + BreamoreBoy
messages: + msg220765
2012-11-05 13:28:59Ramchandra Aptesetversions: + Python 3.4, - Python 3.1, Python 2.7, Python 3.2
2010-07-11 09:48:11BreamoreBoysetversions: + Python 3.1, Python 2.7, Python 3.2
2009-09-27 16:42:50eric.araujosetnosy: + eric.araujo
2009-01-08 22:08:06exarkuncreate