The Python os.getenv() function accesses an Python dictionary which is
mirroring the process environment. This dictionary is populated when the
interpreter starts and updated when os.environ.__setitem__() or
os.putenv() are called. However if the python program imports an
extension module that uses the system putenv() then the changes cannot
be accessed using the Python standard library.

This has been a problem for us as we have created Python bindings to an
existing C based library that modifies the environment dynamically (not
the best design decision...). The workaround we are using is to create
our own wrapper to the system (Solaris/Linux) getenv().

A potential solution could be to make environ a class where
os.environ.__setitem__() calls putenv(), os.environ.__getitem__() calls
getenv() and os.environ.keys()/items()/iter*() uses **environ (or other
appropriate system call). This does however have undefined issues on how
the environment behaves on various systems (memory leaks etc).

I can't see a bug here. If you want the current C library value of the
environment variable, just use os.getenv, not os.environ.

Ignore my comment - I see now that you are talking about os.getenv.

On 9/13/07, Robert Ancell <report@bugs.python.org> wrote:

> The Python os.getenv() function accesses an Python dictionary which is
> mirroring the process environment. This dictionary is populated when the
> interpreter starts and updated when os.environ.__setitem__() or
> os.putenv() are called. However if the python program imports an

As per the document and my simple test (on Linux), os.putenv() does
not update os.environ. I think, it should update it.

> As per the document and my simple test (on Linux), os.putenv() does
> not update os.environ. I think, it should update it.
What would be the benefit ?

> Stefan Sonnenberg-Carstens added the comment:
>
> > As per the document and my simple test (on Linux), os.putenv() does
> > not update os.environ. I think, it should update it.
> What would be the benefit ?

Symmetrical behaviour. When os.getenv() returns the value from
os.environ, one would expect, os.putenv to store the value there (At
least, I did). On the other hand, it is also ok for both os.getenv and
os.putenv get/set the environment directly instead of going through
os.environ. I am sure there was some reason for the current behaviour
of os.putenv. Perhaps, because putenv is supposedly not available on
all platforms?

Any way, I think the OP was asking to always "get" the value
dynamically when ever os.environ['VAR'] or os.getenv['VAR'] is done. I
don't see any problem with that approach. How ever, if it is
considered backwards incompatible, I guess an optional parameter can
be added to os.getenv.

I'd like to see perl/ruby behaviour:
an dict (os.environ), nothing more (perl %ENV,ruby $ENV).
Get rid of setenv/putenv at all.
3.0a1 has even more:
There is os.environ (a dict), os.[put|get]env() and os.environ.putenv()

draghuram, unfortunately while os.putenv() can be fixed to be
symmetrical any putenv call from a C module cannot, for example:

If you make an extension:
#include <stdlib.h>
PyObject *putenvC(PyObject *module, PyObject *args)
{
    int result;

    if (!PyArg_ParseTuple(args, ""))
        return 0;

    result = putenv("FOO=BAR");

    return Py_BuildValue("i", result);
}

The following behaviour will occur:
$ python
>>> import putenv
>>> putenv.putenvC()
>>> assert(os.getenv('FOO') == None)
>>> assert(os.environ.get('FOO') == None)

This is because the os.environ dictionary will never be updated:
From Lib/os.py:
def getenv(key, default=None):
    """Get an environment variable, return None if it doesn't exist.
    The optional second argument can specify an alternate default."""
    return environ.get(key, default)

I've attached proof-of-concept showing how os.environ would ideally
work. It'll only work in Posix, etc etc.

Reading into it more there are a lot of general issues with environments
and memory allocation which is why I suspect Python doesn't use
putenv... See putenv(3) for details.

Compile with:
gcc -shared -o environmodule.so -g -Wall -I /usr/include/python2.5
environmodule.c

> Robert Ancell added the comment:
>
> draghuram, unfortunately while os.putenv() can be fixed to be
> symmetrical any putenv call from a C module cannot, for example:

Hi Robert, I understood the problem from your very first report. I
brought up putenv() not updating os.environ only because you mentioned
in the original report that it does. All you need is a way to get the
current value of an environment variable. The situation is a bit
complicated since a cache is in the picture (os.environ). I would
suggest that you bring up the topic for discussion on python-dev and
once a consensus is reached, some one can come up with a patch (I can
try).

As a workaround, could you use ctypes to pull the environment back into
the python context?  For example:

http://paste.turbogears.org/paste/34734

Can someone please comment on whether or not this issue is still valid.

The problem is the 'putenv' and 'getenv' appear to be parallel, and seem to be documented as being parallel
"Set the environment variable named key to the string value."
"Return the value of the environment variable key if it exists,"
but they are not. Getenv actually looks up the key in the internal os.environ copy while putenv puts to the actual external environment.

This tripped-up someone today on python-list, who did putenv(key, val); getenv(key) and wondered why putenv seemed to have no effect.

I think the solution for this should be to document the asymmetry by adding ' in os.environ' after 'key' in the getenv doc.

--
putenv should not also update os.environ because one can already do that with "os.environ[key] = value" (as recommended) and because the latter uses putenv *if available* to also update the external environment *if possible*. Note that at the time this system was designed, not all systems supported putenv (and perhaps not 'true' getenv either).

getenv(key) is not the same as os.environ[key] because the former has an optional 'default' parameter that defaults to None. So aside from back-compatibility, I do not think the behavior of existing code should change.

A new parameter might be possible. To implement it, one would have to augment the underlying, undocumented, C-coded (for CPython) os-specific module --  posix, nt, os2, ce -- to define a new os-specific getenv function that parallels the os-specific putenv function.

Adding os.environ.update (or .synchronize) to resynchronize os.environ with the external environment would also require a new os-specific function. Currently, the original os.environ is imported as a *data* attribute of the os-specific module.

However, neither of these changes are needed for python code that used os.environ as intended. I don't think we should necessarily cater to badly written C libraries that modify the enviroment in a way that cannot be easily intercepted or controlled. So after making a doc change, I would be inclined to close this pending a python-ideas discussion that supported a new feature.

FYI, PyPy recently got bitten by this: https://bugs.pypy.org/issue1518
A posix.libc_getenv() function could be a solution.