classification
Title: python -m test test_packaging test_zipimport failure
Type: Stage:
Components: Extension Modules, Tests Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: haypo, nadeem.vawda, python-dev, tarek
Priority: normal Keywords: patch

Created on 2011-05-19 20:42 by haypo, last changed 2011-05-21 00:13 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
zipimport_get_data.patch haypo, 2011-05-19 20:42 review
zipimport_get_data-2.patch haypo, 2011-05-19 20:55 review
Messages (5)
msg136334 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-19 20:42
"python -m test test_packaging test_zipimport" fails with:

======================================================================
FAIL: testAFakeZlib (test.test_zipimport.CompressedZipImportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/haypo/prog/HG/cpython/Lib/test/test_zipimport.py", line 130, in testAFakeZlib
    self.fail("expected test to raise ImportError")
AssertionError: expected test to raise ImportError

The problem is that the zipimport module keeps a reference to zlib.decompress() which "makes zlib.decompress immortal": see get_decompress_func() in zipimport.c.

Attached patch replaces the borrowed reference by a classic reference to allow to unload zlib.

I don't think that it makes zipimport slower: it calls PyImport_ImportModuleNoBlock("io") and PyObject_GetAttrString(zlib, "decompress") each time instead of just once, but PyImport_ImportModuleNoBlock() is just a lookup in a dict (if the zlib module is not unloaded between two calls to get_data()).
msg136335 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-19 20:55
OMG! I understood why the bug was not seen before: "python -m test test_zipimport test_zipimport" succeed because test_zipimport REMOVES testAFakeZlib tests after the first run!

New patch to remove this ugly hack (it is no more needed with my patch anyway).
msg136336 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-19 20:57
"test_zipimport REMOVES testAFakeZlib tests after the first run" hack was introduced by a commit supposed to "test zipimport a bit more" :-)

changeset:   36383:7b3d915b6e9d
branch:      legacy-trunk
user:        Neal Norwitz <nnorwitz@gmail.com>
date:        Mon Jan 23 07:52:13 2006 +0000
files:       Lib/test/test_zipimport.py
description:
Test zipimporter a bit more.  Also get working with -R :: option for finding ref leaks
msg136340 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-19 22:22
New changeset 52d9183b34f8 by Victor Stinner in branch '3.1':
Issue #12124: zipimport doesn't keep a reference to zlib.decompress() anymore
http://hg.python.org/cpython/rev/52d9183b34f8

New changeset a043d8e168b3 by Victor Stinner in branch '3.2':
(Merge 3.1) Issue #12124: zipimport doesn't keep a reference to
http://hg.python.org/cpython/rev/a043d8e168b3

New changeset 9e13869b7639 by Victor Stinner in branch 'default':
(Merge 3.2) Issue #12124: zipimport doesn't keep a reference to
http://hg.python.org/cpython/rev/9e13869b7639
msg136412 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-21 00:13
New changeset bfceb8c8178a by Victor Stinner in branch '2.7':
Issue #12124: zipimport doesn't keep a reference to zlib.decompress() anymore
http://hg.python.org/cpython/rev/bfceb8c8178a
History
Date User Action Args
2011-05-21 00:13:53hayposetstatus: open -> closed
resolution: fixed
2011-05-21 00:13:33python-devsetmessages: + msg136412
2011-05-19 22:22:54python-devsetnosy: + python-dev
messages: + msg136340
2011-05-19 20:57:48hayposetmessages: + msg136336
2011-05-19 20:55:10hayposetfiles: + zipimport_get_data-2.patch

messages: + msg136335
2011-05-19 20:54:19nadeem.vawdasetnosy: + nadeem.vawda
2011-05-19 20:42:04haypocreate