This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: pkgutil ImpLoader does not support frozen modules
Type: Stage: resolved
Components: Distutils, Library (Lib) Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Jim.Jewett, dstufft, eric.araujo, lemburg, steve.dower
Priority: normal Keywords: patch

Created on 2014-06-13 14:08 by lemburg, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
pkgutil-frozen-modules-support.patch lemburg, 2014-06-13 17:34 Updated version review
Messages (5)
msg220442 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-06-13 14:08
This leads to problems when using runpy, since it relies on pkgutil to find importers.

In Python 2, ImpLoader is used by ImpImporter which is used as fallback importer by get_importer(). get_importer() is used by runpy to implement e.g. the -m option. As a result, -m doesn't work with frozen modules.

In Python 3, ImpLoader still exists, but is deprecated. get_importer() also no longer falls back to the ImpImporter as default, so the situation is less bothersome.

Here's a patch for Python 3.4.1 which shows how to add support for frozen modules. The same patch also works in Python 2.6 and 2.7.
msg220465 - (view) Author: Jim Jewett (Jim.Jewett) * (Python triager) Date: 2014-06-13 16:49
As best I can tell, this renames the original get_code to _get_code, and then delegates to it after handling the imp.PY_FROZEN case ... why not just add imp.PY_FROZEN to the if/elif chain in the original method?

I also note that the call to self._fix_name doesn't do anything helpful unless it is overridden in a subclass -- was this patch really intended as a recipe for subclasses, that avoided super for backwards compatibility?

If it is intended to go into the main branch, then someone has to decide whether it is OK to enhance despite being deprecated.

If it is just an example, then it should probably be closed with an appropriate status.  I'm guessing "not a bug", but that isn't quite right either.
msg220470 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-06-13 17:19
On 13.06.2014 18:49, Jim Jewett wrote:
> 
> As best I can tell, this renames the original get_code to _get_code, and then delegates to it after handling the imp.PY_FROZEN case ... why not just add imp.PY_FROZEN to the if/elif chain in the original method?

Of course, a proper patch would work that way and I can submit one
if there's demand. The attached patch was the result of a programmatic
patching system that uses search & replace. That's why it uses this
awkward style.

> I also note that the call to self._fix_name doesn't do anything helpful unless it is overridden in a subclass -- was this patch really intended as a recipe for subclasses, that avoided super for backwards compatibility?

It does: if fullname is None, it returns self.fullname.

> If it is intended to go into the main branch, then someone has to decide whether it is OK to enhance despite being deprecated.
> 
> If it is just an example, then it should probably be closed with an appropriate status.  I'm guessing "not a bug", but that isn't quite right either.

It is deprecated in 3.4, but not in 2.7 and for 2.7 I consider this
a bug, since pkgutil claims to support frozen packages in the
ImpImporter doc-string.
msg220471 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-06-13 17:34
Here's an updated patch for Python 3.4.1 that doesn't use the awkward style :-)
msg386401 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-03 18:29
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
History
Date User Action Args
2022-04-11 14:58:04adminsetgithub: 65948
2021-02-03 18:29:01steve.dowersetstatus: open -> closed

nosy: + steve.dower
messages: + msg386401

resolution: out of date
stage: resolved
2014-06-13 17:34:58lemburgsetfiles: - pkgutil-frozen-modules-support.patch
2014-06-13 17:34:28lemburgsetfiles: + pkgutil-frozen-modules-support.patch

messages: + msg220471
2014-06-13 17:19:39lemburgsetmessages: + msg220470
2014-06-13 16:49:28Jim.Jewettsetnosy: + Jim.Jewett
messages: + msg220465
2014-06-13 14:08:31lemburgcreate