classification
Title: urllib/request.py/getproxies_environment() throws "dictionary changed size during iteration" error occasionally
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: alphalfalfa, martin.panter, orsenthil, rhettinger
Priority: normal Keywords:

Created on 2015-11-16 22:46 by alphalfalfa, last changed 2017-03-07 19:26 by serhiy.storchaka. This issue is now closed.

Messages (4)
msg254767 - (view) Author: alphalfalfa (alphalfalfa) Date: 2015-11-16 22:46
For the following lines of code, "dictionary changed size during iteration" would be thrown when the environment variables change while in iteration.

Python-3.5.0/Lib/urllib/request.py
2393     for name, value in os.environ.items():
2394         name = name.lower()
2395         if value and name[-6:] == '_proxy':
2396             proxies[name[:-6]] = value
msg254768 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-16 22:52
How are you managing to get os.environ to change inside that function? Another thread or garbage collector callback perhaps?

My understanding is that os.environ is coupled to the C library’s environment, and that is not thread-safe.
msg254785 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-11-17 07:08
For a simple fix, 2393 should probably read:

    for name, value in list(os.environ.items()):
msg257437 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2016-01-04 02:40
os.environ is never changed by getproxies_environment()

Like Martin Panter, I will curious to know as how the user experienced this issue in the place.

@Raymond Hettinger: If the dict is only read and never modified in the fuction, does it still make sense to have a defensive list() as you suggested? If yes, then there will be couple of places in the stdlib code that will need this change.

```
$ ag os.environ.items Lib
Lib/distutils/_msvccompiler.py
80:            for key, value in os.environ.items()

Lib/test/test_os.py
679:        for key, val in os.environ.items():

Lib/test/test_wsgiref.py
427:    os_environ = dict(os.environ.items())

Lib/urllib/request.py
2453:    for name, value in os.environ.items():

Lib/wsgiref/handlers.py
47:    for k, v in os.environ.items():
```

My understanding is, if the function is not manipulation and only reading the dictionary, a defensive list or copy is not required. And, this bug report should be closed.
History
Date User Action Args
2017-03-07 19:26:21serhiy.storchakasetstatus: pending -> closed
resolution: not a bug
stage: test needed -> resolved
2016-01-04 02:40:13orsenthilsetstatus: open -> pending
nosy: + orsenthil
messages: + msg257437

2015-11-17 07:08:26rhettingersetnosy: + rhettinger
messages: + msg254785
2015-11-16 22:52:31martin.pantersetnosy: + martin.panter
messages: + msg254768

type: behavior
stage: test needed
2015-11-16 22:46:08alphalfalfacreate