This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: pkgutil.walk_packages jumps out from given path if there is package with the same name in sys.path
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: karkucik, ncoghlan
Priority: normal Keywords: patch

Created on 2019-02-20 16:37 by karkucik, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 11956 open karkucik, 2019-02-20 16:44
Messages (6)
msg336111 - (view) Author: Piotr Karkut (karkucik) * Date: 2019-02-20 16:37
When walk_packages encounter a package with a name that is available in sys.path, it will abandon the current package, and start walking the package from the sys.path.

Consider this file layout:

|   ├──core   
|   |   ├──some_package/
|   |   |   ├──
|   |   |   └──
|   |   └──
|   └──
   |   ├──
   |   └──

The result of walking package1 will be:

>> pkgutil.walk_packages('PYTHONPATH/package1')

ModuleInfo(module_finder=FileFinder('PYTHONPATH/package1/core'), name='some_package', ispkg=True)
ModuleInfo(module_finder=FileFinder('PYTHONPATH/some_package), name='another_mod', ispkg=False)

I'm not sure if it is a security issue, but it definitely should not jump off the given path.
msg339028 - (view) Author: Piotr Karkut (karkucik) * Date: 2019-03-28 10:06
msg339573 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2019-04-07 11:52
Piotr: does it always jump out, or does it only jump out if the relevant module has already been imported?

(The tests for walk_packages are relatively weak and never generate conflicting names, so it's entirely plausible that there are caching side effects that make it do strange things)
msg339603 - (view) Author: Piotr Karkut (karkucik) * Date: 2019-04-08 09:14
Nick: From what I've checked, it jumps in case the module is already imported. The problem is that the original implementation is quite naive, and it's trying to import the module before looking for it in `sys.modules` - So if the module with a conflicting name is in PYTHONPATH and has higher priority, it'd be imported instead of the correct one. And then, as the module is imported, it'd be available in `sys.modules`.
msg343747 - (view) Author: Piotr Karkut (karkucik) * Date: 2019-05-28 08:22
msg348381 - (view) Author: Piotr Karkut (karkucik) * Date: 2019-07-24 12:03
Date User Action Args
2022-04-11 14:59:11adminsetgithub: 80234
2019-07-24 12:03:00karkuciksetmessages: + msg348381
2019-05-28 08:22:35karkuciksetmessages: + msg343747
2019-04-08 09:14:06karkuciksetmessages: + msg339603
2019-04-07 11:52:51ncoghlansetmessages: + msg339573
2019-04-03 08:44:42SilentGhostsetnosy: + ncoghlan

versions: - Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7
2019-03-28 10:06:49karkuciksetmessages: + msg339028
2019-02-20 16:44:56karkuciksetkeywords: + patch
stage: patch review
pull_requests: + pull_request11982
2019-02-20 16:42:04karkuciksettitle: pkgutil.walk_packages jumps out from given path if there is package with the same name in sys.pah -> pkgutil.walk_packages jumps out from given path if there is package with the same name in sys.path
2019-02-20 16:37:07karkucikcreate