This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: subprocess makes environment blocks with duplicate keys on Windows
Type: behavior Stage:
Components: Library (Lib), Windows Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benrg, eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2022-02-26 04:19 by benrg, last changed 2022-04-11 14:59 by admin.

Messages (3)
msg414068 - (view) Author: (benrg) Date: 2022-02-26 04:19
On Windows, if one writes

    env = os.environ.copy()
    env['http_proxy'] = 'whatever'

or either of the documented equivalents ({**os.environ, ...} or (os.environ | {...})), and passes the resulting environment to subprocess.run or subprocess.Popen, the spawned process may get an environment containing both `HTTP_PROXY` and `http_proxy`. Most Win32 software will see only the first one, which contains the unmodified value from os.environ.

Because os.environ forces all keys to upper case, it's possible to work around this by using only upper case keys in the update, but that behavior of os.environ is nonstandard (issue 46861), and subprocess shouldn't depend on it always being true, nor should end users have to.

Since dicts preserve order, the user's (presumable) intent is preserved in the env argument. I think subprocess should do something like

    env = {k.upper(): (k, v) for k, v in env.items()}
    env = dict(env.values())

to discard duplicate keys, keeping only the rightmost one.
msg414074 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-02-26 06:00
This should be handled in _winapi.CreateProcess(). An environment block is technically required to be sorted. (Ages ago this was a MUST requirement for getting and setting variables to work correctly, since the implementation depended on the sort order, but I think nowadays it's a SHOULD requirement.) For example, see the documentation of CreateProcessW() [1]:

    If an application provides an environment block, ... explicitly create
    these environment variable strings, sort them alphabetically (because
    the system uses a sorted environment)

"Changing Environment Variables" is more specific [2]:

    All strings in the environment block must be sorted alphabetically by name.
    The sort is case-insensitive, Unicode order, without regard to locale.

CompareStringOrdinal() [3] implements a case-insensitive ordinal comparison. When a key compares as equal, either keep the current one in the sorted list, or replace it with the new key.

---

[1] https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessw
[2] https://docs.microsoft.com/en-us/windows/win32/procthread/changing-environment-variables
[3] https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-comparestringordinal
msg414075 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-02-26 06:07
I suggest closing this issue as a duplicate of bpo-43702.
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 91018
2022-02-26 06:07:22eryksunsetmessages: + msg414075
2022-02-26 06:00:31eryksunsetnosy: + eryksun

messages: + msg414074
versions: + Python 3.9, Python 3.11
2022-02-26 04:19:39benrgcreate