Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkgutil.iter_zipimport_modules ignores the prefix parameter for packages #58417

Closed
JamesPickering mannequin opened this issue Mar 6, 2012 · 7 comments
Closed

pkgutil.iter_zipimport_modules ignores the prefix parameter for packages #58417

JamesPickering mannequin opened this issue Mar 6, 2012 · 7 comments
Assignees
@ambv
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@JamesPickering
Copy link
Mannequin

JamesPickering mannequin commented Mar 6, 2012

BPO 14209
Nosy @brettcannon, @pjeby, @ncoghlan, @ambv, @ericsnowcurrently

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/ambv'
closed_at = <Date 2016-06-12.01:09:36.667>
created_at = <Date 2012-03-06.15:02:24.250>
labels = ['type-bug', 'library']
title = 'pkgutil.iter_zipimport_modules ignores the prefix parameter for packages'
updated_at = <Date 2016-06-12.01:09:36.379>
user = 'https://bugs.python.org/JamesPickering'

bugs.python.org fields:

activity = <Date 2016-06-12.01:09:36.379>
actor = 'lukasz.langa'
assignee = 'lukasz.langa'
closed = True
closed_date = <Date 2016-06-12.01:09:36.667>
closer = 'lukasz.langa'
components = ['Library (Lib)']
creation = <Date 2012-03-06.15:02:24.250>
creator = 'James.Pickering'
dependencies = []
files = []
hgrepos = []
issue_num = 14209
keywords = []
message_count = 7.0
messages = ['155018', '155137', '265221', '265223', '265241', '268306', '268307']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'pje', 'ncoghlan', 'lukasz.langa', 'python-dev', 'eric.snow', 'James.Pickering']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'test needed'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14209'
versions = ['Python 3.5', 'Python 3.6']
@JamesPickering
Copy link
Mannequin Author

JamesPickering mannequin commented Mar 6, 2012

If you run pkgutil.iter_zipimport_modules with a prefix parameter, and the module in question is a package, then the prefix parameter is ignored.

The most visible symptom of this is when running pkgutil.walk_packages for a zipfile. Imagine we have a module structure like this (or create one):

a/
a/init.py
a/b/
a/b/__init.__py

If we put this structure in a directory, add the directory to sys.path, and run pkgutil.walk_packages(), it will find modules "a" and "a.b".

If we put this structure in a zipfile, however, we add this file to sys.path, and run pkgutil.walk_packages(), it will find modules "a" and "b". This is because pkgutil.iter_zipimport_modules ignores the prefix parameter "a.".

This is incorrect.

This can be fixed by changing line ~344 of Lib/pkgutil.py from:

yield fn[0], True

to

yield prefix + fn[0], True

Thanks, James

P.s, This is my first Python bug report. I apologise in advance for any poor etiquette.

@JamesPickering JamesPickering mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 6, 2012
@ncoghlan
Copy link
Contributor

ncoghlan commented Mar 8, 2012

Adding Brett, since the plan is to clear out a lot of the redundant code in pkgutil once importlib is fully bootstrapped as the standard import implementation.

(although this will still affect the older versions directly)

@ambv
Copy link
Contributor

ambv commented May 10, 2016

Brett, Facebook is using the proposed patch in prod since last year. It works fine. Maybe we should just include it for the time being?

@brettcannon
Copy link
Member

I don't deal with pkgutil so I have no problem if you want to go ahead and apply the patch if you think it's reasonable to accept, Łukasz.

@ncoghlan
Copy link
Contributor

+1 for just including the fix in the next round of maintenance releases, although a test case would be desirable.

@python-dev
Copy link
Mannequin

python-dev mannequin commented Jun 12, 2016

New changeset 9649acf7d472 by Łukasz Langa in branch '3.5':
Issue bpo-14209: pkgutil.iter_zipimport_modules ignores the prefix for packages
https://hg.python.org/cpython/rev/9649acf7d472

New changeset 389b7456a053 by Łukasz Langa in branch 'default':
Merge 3.5, issue bpo-14209
https://hg.python.org/cpython/rev/389b7456a053

@ambv
Copy link
Contributor

ambv commented Jun 12, 2016

Done. Fix is going to be present for 3.5.2 and 3.6. Thank you, James.

@ambv ambv closed this as completed Jun 12, 2016
@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ambv ambv

Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@brettcannon @ambv @ncoghlan