classification
Title: test_site/test_startup_imports fails when mpl_toolkit or logilab based modules installed
Type: Stage:
Components: Tests Versions: Python 3.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, brett.cannon, christian.heimes, drudd, ncoghlan, vinay.sajip
Priority: normal Keywords:

Created on 2015-01-20 04:48 by drudd, last changed 2015-03-08 14:58 by brett.cannon. This issue is now closed.

Messages (3)
msg234345 - (view) Author: Douglas Rudd (drudd) Date: 2015-01-20 04:48
pth files for logilab (e.g. logilab_common, logilab_astng) and mpl_toolkit (e.g. basemap, matplotlib) contain code like the following (taken from basemap 1.0.7):

import sys, types, os;p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('mpl_toolkits',));ie = os.path.exists(os.path.join(p,'__init__.py'));m = not ie and sys.modules.setdefault('mpl_toolkits', types.ModuleType('mpl_toolkits'));mp = (m or []) and m.__dict__.setdefault('__path__',[]);(p not in mp) and mp.append(p)

This leads to the types module being loaded on import, which then causes test_site/test_startup_imports to fail since types is in collection_mods:

        collection_mods = {'_collections', 'collections', 'functools',
                           'heapq', 'itertools', 'keyword', 'operator',
                           'reprlib', 'types', 'weakref'
                          }.difference(sys.builtin_module_names)
        self.assertFalse(modules.intersection(collection_mods), stderr)

$ python3.4 -c 'import sys; print(set(sys.modules))'
{'builtins', '_codecs', 'io', 'errno', 'abc', '_weakref', '_collections_abc', '_bootlocale', 'stat', 'os', '_frozen_importlib', 'encodings.utf_8', '_sitebuiltins', '_sysconfigdata', 'sysconfig', 'posixpath', 'sys', 'mpl_toolkits', '_weakrefset', 'encodings', 'encodings.aliases', 'signal', '__main__', '_stat', 'zipimport', 'genericpath', 'site', '_io', 'posix', 'codecs', 'types', '_imp', 'os.path', '_warnings', 'marshal', '_locale', '_thread', 'encodings.latin_1'}


This is a similar bug to http://bugs.python.org/issue20986 

I don't know the purpose of this test well enough to propose a fix, but given the number of exceptions already there it doesn't seem to be a robust test.
msg237471 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-03-07 19:36
I've added names from the nosy list for #20986 as it appears to be similar.
msg237536 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-03-08 14:58
Python's startup performance is very sensitive to what modules get imported for all Python programs. The tests that are failing for you make sure that a core developer doesn't accidentally introduce a new startup dependency and thus slow startup times without doing so purposefully. So the tests are working as intended, you unfortunately just happen to have a startup environment thanks to those .pth files which trigger the failures.
History
Date User Action Args
2015-03-08 14:58:25brett.cannonsetstatus: open -> closed
resolution: not a bug
messages: + msg237536
2015-03-07 19:36:52BreamoreBoysetnosy: + christian.heimes, brett.cannon, ncoghlan, BreamoreBoy, vinay.sajip
messages: + msg237471
2015-01-20 04:48:28druddcreate