Author martin.panter
Recipients brett.cannon, martin.panter, r.david.murray
Date 2015-11-19.03:08:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1447902491.23.0.411223484207.issue25533@psf.upfronthosting.co.za>
In-reply-to
Content
I did some work on adding support for frozen modules, but I got stuck. The low level is fine:

>>> pprint(sys.get_frozen_modules())  # (name, ispkg) pairs
(('_frozen_importlib', False),
 ('_frozen_importlib_external', False),
 ('__hello__', False),
 ('__phello__', True),
 ('__phello__.spam', False))
>>> print("\n".join(map(repr, pkgutil.iter_modules(builtins=True))))
(<class '_frozen_importlib.BuiltinImporter'>, '_ast', False)
. . .
(<class '_frozen_importlib.BuiltinImporter'>, 'zipimport', False)
(<class '_frozen_importlib.FrozenImporter'>, '_frozen_importlib', False)
(<class '_frozen_importlib.FrozenImporter'>, '_frozen_importlib_external', False)
(<class '_frozen_importlib.FrozenImporter'>, '__hello__', False)
(<class '_frozen_importlib.FrozenImporter'>, '__phello__', True)
. . .
(FileFinder('.'), 'python-config', False)
. . .

But the current __hello__ and __phello__ modules print stuff when you import them, which messes with walk_packages(), pydoc, etc:

$ ./python -m pydoc -k pkgutil
Hello world!
Hello world!
Hello world!
pkgutil - Utilities to support packages.
test.test_pkgutil 

When I stopped these frozen modules from printing on import (as in my current patch), I found this broke the test suite. In particular, test_importlib.frozen.test_loader relies on the printouts to test what gets executed when importing frozen modules. So I am not sure the best way to continue if I am to add support for frozen modules to iter_modules().

Another problem was that there is no way to list submodules of a frozen package unless you know the package’s name. Currently, iter_modules() only sees a path list, which is empty for __phello__. However, I was able to add a special case to walk_packages(None) to include frozen submodules.

Some questions:

1. Do people think the general idea of enhancing iter_modules() is worthwhile?

2. Should I try harder to solve the problem with running frozen modules, or is it sensible to just leave out the frozen module support?
History
Date User Action Args
2015-11-19 03:08:12martin.pantersetrecipients: + martin.panter, brett.cannon, r.david.murray
2015-11-19 03:08:11martin.pantersetmessageid: <1447902491.23.0.411223484207.issue25533@psf.upfronthosting.co.za>
2015-11-19 03:08:11martin.panterlinkissue25533 messages
2015-11-19 03:08:10martin.pantercreate