classification
Title: Remove inconsistent behavior between import and zipimport
Type: Stage: resolved
Components: Interpreter Core Versions:
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: acidbase, brett.cannon, eric.araujo, loewis, ncoghlan
Priority: normal Keywords: patch

Created on 2005-11-03 03:43 by acidbase, last changed 2012-04-22 17:14 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
patch_import.diff acidbase, 2005-11-03 03:43 Patch to 'fix' this inconsitent behavior import/zipimport
Messages (11)
msg48962 - (view) Author: Osvaldo Santana Neto (acidbase) Date: 2005-11-03 03:43
Look the inconsistent behavior:
$ ls
modulo_c.pyc  modulo_o.pyo
$ zip modulos.zip modulo_o.pyo modulo_c.pyc
  adding: modulo_o.pyo (deflated 38%)
  adding: modulo_c.pyc (deflated 38%)
$ ls
modulo_c.pyc  modulo_o.pyo  modulos.zip

$ python2.4
>>> import modulo_c
modulo_c
>>> import modulo_o
ImportError: No module named modulo_o

$ python2.4 -O
>>> import modulo_c
ImportError: No module named modulo_c
>>> import modulo_o
modulo_o

$ rm *.pyc *.pyo
$ ls
modulos.zip
$ PYTHONPATH=modulos.zip python2.4
>>> import modulo_c
modulo_c
>>> import modulo_o
modulo_o
$ PYTHONPATH=modulos.zip python2.4 -O
>>> import modulo_c
modulo_c
>>> import modulo_o
modulo_o
msg48963 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-11-05 03:15
Logged In: YES 
user_id=21627

The discussion on python-dev (both last year's and this
year's) has shown that this is the wrong way of fixing the
inconsistency. Instead, zipfile should stop importing .pyo
files when -O isn't given. Rejecting the patch as invalid.
msg154681 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-01 08:45
Still an issue with 3.2: zipimport considers both pyc and pyo with or without -O.

The discussion starts here: http://mail.python.org/pipermail/python-dev/2006-November/thread.html#69822

(This was originally closed instead of asking for another fix because SourceForge separated bugs and patches.)
msg154682 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-01 08:48
I need to qualify something: I reproduced the bug with legacy/pre-PEP 3147/in-the-same-dir pyc and pyo files.  One could argue that pycache directories in 3.2+ make this irrelevant and that it’s too late for 2.x.
msg154698 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-03-01 15:28
I like that argument. =) If this is not an issue in Python 3.3 then this should be closed as out of date since it will break code if it is changed.
msg154703 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-01 16:29
It entirely depends on how much you care about pyc-only/pyo-only zipfile distributions, and compatibility between zipimport and importlib (i.e. if you don’t plan on matching the zipimport bug in importlib, might as well fix zipimport in 2.7 and 3.2).
msg154711 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-03-01 19:47
I don't care about compatibility between zipimport and importlib.
msg154733 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-02 05:38
OK, I’m leaving this open until the next weekly report just in case someone interested comes here and weighs in, otherwise I’ll close as wontfix.
msg154738 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-03-02 05:49
Does "compileall" generate both .pyc and .pyo by default? Or do you have to run it twice? If the latter, does pysetup handle that for you?

MvL is correct that zipimport should ignore .pyo files when __debug__ is set and vice-versa, but the precompilation tools should also take care of generating both versions by default.
msg154739 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-02 05:54
> Does "compileall" generate both .pyc and .pyo by default?
python3 -m compileall generates pyc, python3 -O -m compileall pyo.  Functions in py_compile and compileall gained an optimize argument in 3.2.

> does pysetup handle that for you?
You’ll have to be more specific.  Bytecode files can be created when building a bdist or on install from source.  Packaging commands do not depend on the calling Python’s -O option, they have their own options to let users specify if they want pyc files, pyo, neither or both.

> MvL is correct that zipimport should ignore .pyo files when __debug__ is set and vice-versa, but the
> precompilation tools should also take care of generating both versions by default.
Really?  The behaviors of compileall and packaging seems better to me.
msg154741 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-03-02 06:37
Status quo sounds fine then. +1 for closing it again.
History
Date User Action Args
2012-04-22 17:14:45eric.araujosetstatus: open -> closed
stage: needs patch -> resolved
resolution: wont fix
versions: - Python 2.7, Python 3.2, Python 3.3
2012-03-02 06:37:46ncoghlansetmessages: + msg154741
2012-03-02 05:54:06eric.araujosetmessages: + msg154739
2012-03-02 05:49:57ncoghlansetmessages: + msg154738
2012-03-02 05:38:24eric.araujosetmessages: + msg154733
2012-03-01 19:47:40brett.cannonsetmessages: + msg154711
2012-03-01 16:29:08eric.araujosetmessages: + msg154703
2012-03-01 15:28:01brett.cannonsetmessages: + msg154698
2012-03-01 08:48:02eric.araujosetmessages: + msg154682
2012-03-01 08:45:12eric.araujosetstatus: closed -> open

versions: + Python 2.7, Python 3.2, Python 3.3, - Python 2.5
nosy: + brett.cannon, ncoghlan, eric.araujo

messages: + msg154681
resolution: not a bug -> (no value)
stage: needs patch
2005-11-03 03:43:17acidbasecreate