classification
Title: imp.py: load_package() appends to its own loop variable
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Alexandru Ardelean, SylvainDe, brett.cannon, eric.snow, ncoghlan, r.david.murray
Priority: normal Keywords:

Created on 2017-06-13 08:25 by Alexandru Ardelean, last changed 2017-06-23 18:40 by brett.cannon. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2268 merged Alexandru Ardelean, 2017-06-18 16:45
PR 2364 merged brett.cannon, 2017-06-23 17:45
PR 2365 merged brett.cannon, 2017-06-23 17:48
Messages (12)
msg295865 - (view) Author: Alexandru Ardelean (Alexandru Ardelean) * Date: 2017-06-13 08:25
On OpenWrt, the Python & Python3 are shipped with bytecodes [also, bytecode compiling is disabled by default].
2 reasons for that: 1) that would fill up flash/RAM 2) bytecodes can be up to 10x faster than Python source code

I got a report of an issue with virtualenv on Python3, that had a weird path: the `virtualenv /test` would generate `/test/lib/virtualenv/__init__.py/__init__.pyc`


The fix is here in the patch file:
https://github.com/openwrt/packages/pull/4475/commits/576c45eb0a5314121aeb2f8d8931644b65e5be99
msg295964 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-06-13 22:20
Three things.

One, the imp module is deprecated, so please don't use it.

Two, all undocumented functions in the imp module are considered unsupported, so double-please don't use imp.load_package(). :)

Three, we can't look at that patch without the person supplying it signing Python's CLA as we risk running afoul of things legally.
msg295991 - (view) Author: Alexandru Ardelean (Alexandru Ardelean) * Date: 2017-06-14 07:42
Damn.

You guys are more corporate than... <scratching on forehead>... anyway.

Please tell me, how to send you my signed CLA.
Not sure if posting the link or PDF is fine here, but I'd be fine with that too.

To resolve your first 2 notes: I am not using the imp module, nor the imp.load_package() function.
I did see it was marked DEPRECATED.

But, virtualenv uses it in it's latest release [15.1.0].
This is from trunk:
https://github.com/pypa/virtualenv/blob/master/virtualenv_embedded/distutils-init.py#L17

But that's not the actual code that's used.
Seems virtualenv, has a twisted way of keeping Python code base64 encoded & zipped within the virtualenv.py script.
https://github.com/pypa/virtualenv/blob/master/virtualenv.py#L2117

So, maybe we can tell them not to use it, but let's see.
In my case, I just have a bug report [I was notified via email directly], that this is an issue with Python3 + virtualenv on LEDE & OpenWrt.
Python + virtualenv is not affected.

While I'm here, maybe I could use this opportunity to start sending a few more patches from the OpenWrt/LEDE package build.
But let's see how we resolve the legal matter first.
msg295992 - (view) Author: SylvainDe (SylvainDe) * Date: 2017-06-14 07:50
Alexandru: You'll find the PSF Contributor Agreement at https://www.python.org/psf/contrib/contrib-form/ .
msg295993 - (view) Author: Alexandru Ardelean (Alexandru Ardelean) * Date: 2017-06-14 07:55
I already completed those steps.
Question is: what do I do with the output from that eSign ?
Or, how do I use it ?

It only mentions "Finally, click the "eSign" button to have the form sent to the PSF Secretary, Ewa Jodlowska."

It's unclear from there-on if my patch is acceptable, or whether I can submit it [or anything], wait for a reply ; basically, how to continue once the form is signed.
msg296012 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-06-14 15:32
Yes, that's all you needed to do.  She updated your status in the tracker (you now have the CLA signed '*' next  to your name).
msg296038 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-06-14 20:49
Yes, we're as corporate as an open source project that doesn't have enough funding to be able to defend against a lawsuit but is big enough that simply shutting down would really suck for millions of people. ;) Anyway, your CLA signing came through so now we just need a pull request on GitHub or a patch.

As for virtualenv still using a deprecated, hidden API, that's bad on them and should probably be reported to them directly.
msg296278 - (view) Author: Alexandru Ardelean (Alexandru Ardelean) * Date: 2017-06-18 17:00
FWIW, I took a look over the virtualenv repo.
There's an issue raised there
https://github.com/pypa/virtualenv/issues/955

And a PR to address it 
https://github.com/pypa/virtualenv/pull/947

Though, it seems like they still want to keep `imp.load_package()` for a while, in order to keep compatibility with Python 2.6.
msg296279 - (view) Author: Alexandru Ardelean (Alexandru Ardelean) * Date: 2017-06-18 17:03
Admittedly, that PR refers to `imp.load_module()`
But, `imp.load_modules()` also calls `imp.load_package()`
msg296730 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-06-23 17:35
New changeset c38e32a10061a7c6d54e7e53ffabf7af7998f045 by Brett Cannon (Alexandru Ardelean) in branch 'master':
bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268)
https://github.com/python/cpython/commit/c38e32a10061a7c6d54e7e53ffabf7af7998f045
msg296733 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-06-23 18:23
New changeset 9db3ae045dd462a2da2e016c44231de1befd1f87 by Brett Cannon in branch '3.6':
[3.6] bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268) (#2364)
https://github.com/python/cpython/commit/9db3ae045dd462a2da2e016c44231de1befd1f87
msg296734 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-06-23 18:39
New changeset 599ff020b308113f3709fd4e623d9f0d08511706 by Brett Cannon in branch '3.5':
[3.5] bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268) (GH-2365)
https://github.com/python/cpython/commit/599ff020b308113f3709fd4e623d9f0d08511706
History
Date User Action Args
2017-06-23 18:40:28brett.cannonsetstatus: open -> closed
stage: resolved
resolution: fixed
versions: + Python 3.5
2017-06-23 18:39:56brett.cannonsetmessages: + msg296734
2017-06-23 18:23:39brett.cannonsetmessages: + msg296733
2017-06-23 17:48:23brett.cannonsetpull_requests: + pull_request2414
2017-06-23 17:45:57brett.cannonsetpull_requests: + pull_request2413
2017-06-23 17:35:05brett.cannonsetmessages: + msg296730
2017-06-18 17:03:39Alexandru Ardeleansetmessages: + msg296279
2017-06-18 17:00:17Alexandru Ardeleansetmessages: + msg296278
2017-06-18 16:45:32Alexandru Ardeleansetpull_requests: + pull_request2318
2017-06-14 20:49:32brett.cannonsetmessages: + msg296038
2017-06-14 15:32:42r.david.murraysetmessages: + msg296012
2017-06-14 07:55:10Alexandru Ardeleansetmessages: + msg295993
2017-06-14 07:50:22SylvainDesetnosy: + SylvainDe
messages: + msg295992
2017-06-14 07:42:44Alexandru Ardeleansetmessages: + msg295991
2017-06-13 22:21:18brett.cannonsettitle: imp.py: load_package: function has a buggy loop with `path = os.path.join(path, '__init__'+extension)` -> imp.py: load_package() appends to its own loop variable
2017-06-13 22:20:38brett.cannonsetmessages: + msg295964
2017-06-13 21:44:03r.david.murraysetnosy: + brett.cannon, ncoghlan, r.david.murray, eric.snow
2017-06-13 08:27:03Alexandru Ardeleansettype: behavior
2017-06-13 08:25:25Alexandru Ardeleancreate