Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

urllib/request.py/getproxies_environment() throws "dictionary changed size during iteration" error occasionally #69827

Closed
alphalfalfa mannequin opened this issue Nov 16, 2015 · 4 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@alphalfalfa
Copy link
Mannequin

alphalfalfa mannequin commented Nov 16, 2015

BPO 25641
Nosy @rhettinger, @orsenthil, @vadmium

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-03-07.19:26:21.887>
created_at = <Date 2015-11-16.22:46:08.810>
labels = ['invalid', 'type-bug', 'library']
title = 'urllib/request.py/getproxies_environment() throws "dictionary changed size during iteration" error occasionally'
updated_at = <Date 2017-03-07.19:26:21.886>
user = 'https://bugs.python.org/alphalfalfa'

bugs.python.org fields:

activity = <Date 2017-03-07.19:26:21.886>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-03-07.19:26:21.887>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2015-11-16.22:46:08.810>
creator = 'alphalfalfa'
dependencies = []
files = []
hgrepos = []
issue_num = 25641
keywords = []
message_count = 4.0
messages = ['254767', '254768', '254785', '257437']
nosy_count = 4.0
nosy_names = ['rhettinger', 'orsenthil', 'martin.panter', 'alphalfalfa']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25641'
versions = ['Python 3.4', 'Python 3.5']

@alphalfalfa
Copy link
Mannequin Author

alphalfalfa mannequin commented Nov 16, 2015

For the following lines of code, "dictionary changed size during iteration" would be thrown when the environment variables change while in iteration.

Python-3.5.0/Lib/urllib/request.py
2393 for name, value in os.environ.items():
2394 name = name.lower()
2395 if value and name[-6:] == '_proxy':
2396 proxies[name[:-6]] = value

@alphalfalfa alphalfalfa mannequin added the stdlib Python modules in the Lib dir label Nov 16, 2015
@vadmium
Copy link
Member

vadmium commented Nov 16, 2015

How are you managing to get os.environ to change inside that function? Another thread or garbage collector callback perhaps?

My understanding is that os.environ is coupled to the C library’s environment, and that is not thread-safe.

@vadmium vadmium added the type-bug An unexpected behavior, bug, or error label Nov 16, 2015
@rhettinger
Copy link
Contributor

For a simple fix, 2393 should probably read:

    for name, value in list(os.environ.items()):

@orsenthil
Copy link
Member

os.environ is never changed by getproxies_environment()

Like Martin Panter, I will curious to know as how the user experienced this issue in the place.

@Raymond Hettinger: If the dict is only read and never modified in the fuction, does it still make sense to have a defensive list() as you suggested? If yes, then there will be couple of places in the stdlib code that will need this change.

$ ag os.environ.items Lib
Lib/distutils/_msvccompiler.py
80:            for key, value in os.environ.items()

Lib/test/test_os.py
679:        for key, val in os.environ.items():

Lib/test/test_wsgiref.py
427:    os_environ = dict(os.environ.items())

Lib/urllib/request.py
2453:    for name, value in os.environ.items():

Lib/wsgiref/handlers.py
47:    for k, v in os.environ.items():

My understanding is, if the function is not manipulation and only reading the dictionary, a defensive list or copy is not required. And, this bug report should be closed.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

4 participants