classification
Title: inspect.getmodule fails when module imports change sys.modules
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: Erik.Tollerud, amaury.forgeotdarc, eric.araujo, eric.snow, meador.inge, python-dev
Priority: normal Keywords: patch

Created on 2011-11-27 03:06 by Erik.Tollerud, last changed 2011-11-29 16:19 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
getmodulefix.patch Erik.Tollerud, 2011-11-27 03:06 review
Messages (7)
msg148438 - (view) Author: Erik Tollerud (Erik.Tollerud) Date: 2011-11-27 03:06
The inspect.getmodule function crashes if packages are installed that futz with sys.modules while they are being tested for module status or the like.  I'm not actually sure which packages are doing this, but the symptom is the for loop over sys.modules raises an Exception because it is modified while the loop is running.

This is *not* a problem in Python 2.x because sys.modules.items() returns a copy of the dictionary instead of an iterator, and 3.x changes that behavior.  The comment above the for loop makes it clear that the expected behavior is a copy rather than an iterator, so the attached patch corrects the problem by simply wrapping the items() call in list().
msg148459 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-11-27 21:46
You are certainly right, but I wonder how this can happen.
Are there modules which import something just by looking at them?
Or is is some race condition due to another running thread?
msg148487 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-28 15:08
Maybe it can be caused by an installation happening during the loop.  I agree with Erik’s reading of the comment and patch, and don’t think a test is needed.
msg148509 - (view) Author: Erik Tollerud (Erik.Tollerud) Date: 2011-11-28 18:37
The package that triggers it for me is the py (http://pypi.python.org/pypi/py) package - when in gets imported, it does some trick with sys.modules that is in place to get around some pickling restriction, but that means sys.modules is altered during the import of the `py` package... and that triggers the exception.
msg148511 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-11-28 18:52
When a package is imported sys.modules changes... nothing special here.
But it's true true that py.std, for example, is a "lazy" module with a special __getattr__ that will import submodules.

Patch looks good to me as well.
msg148581 - (view) Author: Roundup Robot (python-dev) Date: 2011-11-29 16:15
New changeset 2ef359d7a2e9 by Éric Araujo in branch '3.2':
Fix inspect.getmodule to use a copy of sys.modules for iteration (#13487).
http://hg.python.org/cpython/rev/2ef359d7a2e9
msg148583 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-29 16:19
Committed, thanks.
History
Date User Action Args
2011-11-29 16:19:19eric.araujosetstatus: open -> closed
resolution: fixed
messages: + msg148583

stage: patch review -> resolved
2011-11-29 16:15:06python-devsetnosy: + python-dev
messages: + msg148581
2011-11-29 15:37:31eric.araujosetassignee: eric.araujo
2011-11-28 18:52:47amaury.forgeotdarcsetmessages: + msg148511
2011-11-28 18:37:46Erik.Tollerudsetmessages: + msg148509
2011-11-28 15:08:04eric.araujosetversions: - Python 3.1
nosy: + eric.araujo

messages: + msg148487

type: behavior
stage: patch review
2011-11-27 21:46:56amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg148459
2011-11-27 17:18:55meador.ingesetnosy: + meador.inge
2011-11-27 03:13:56eric.snowsetnosy: + eric.snow
2011-11-27 03:06:48Erik.Tollerudcreate