classification
Title: site ignores ImportError when running sitecustomize and usercustomize
Type: enhancement Stage:
Components: Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: haypo Nosy List: brett.cannon, haypo, python-dev
Priority: normal Keywords: patch

Created on 2016-01-13 14:46 by haypo, last changed 2016-01-22 11:23 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
site.patch haypo, 2016-01-13 14:46
Messages (7)
msg258144 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-01-13 14:46
If the code of sitecustomize raises an ImportError because the requested module doesn't exist, sitecustomize exception is silently ignored because site.py uses "try: import sitecustomize except ImportError: pass".

It's possible to log a warning since ImportError now has a name attribute since Python 3.3.

There is a similar issue on usercustomize.

Attached patch fixes the issue.
msg258160 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-13 17:44
This is a change in semantics. It might be better to log an ImportWarning when the import fails and keep the current semantics (and be careful about importing warnings).
msg258510 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-01-18 09:12
Example of the bug:

----
$ cat Lib/sitecustomize.py
print("before")
import xxx
print("after")

haypo@smithers$ ./python -q -c pass
before
----

The line "after" is not executed and *no* error nor warning is raised. With the patch:

----
$ ./python -q -c pass
before
Error in sitecustomize; set PYTHONVERBOSE for traceback:
ImportError: No module named 'xxx'

$ PYTHONVERBOSE=1 ./python -q -c pass
(...)
# /home/haypo/prog/python/3.5/Lib/__pycache__/sitecustomize.cpython-35.pyc matches /home/haypo/prog/python/3.5/Lib/sitecustomize.py
# code object from '/home/haypo/prog/python/3.5/Lib/__pycache__/sitecustomize.cpython-35.pyc'
before
Traceback (most recent call last):
  File "/home/haypo/prog/python/3.5/Lib/site.py", line 508, in execsitecustomize
    import sitecustomize
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 662, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/haypo/prog/python/3.5/Lib/sitecustomize.py", line 2, in <module>
    import xxx
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'xxx'
# destroy sitecustomize
(...)
----

Brett Cannon: "This is a change in semantics."

Can you please elaborate? Even if importing sitecustomize changes, Python continue its execution. Only a warning is emited.

Brett Cannon: "It might be better to log an ImportWarning when the import fails and keep the current semantics (and be careful about importing warnings)."

With my patch, a message "Error in sitecustomize; set PYTHONVERBOSE for traceback:" is logged. I used the same behaviour then the code to handle site import.

Note: I found this issue when I worked on the PEP 511 to register a code transformer at startup. The bug is really annoying: the code transformers may or may not be registered depending if required modules can be important, I expect an error (or at least a warning).
msg258758 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-01-21 16:59
@Brett: Ping? I'm going to push the fix to Python 3.6.

Tell me if you are ok to backport the change to Python 2.7 and 3.5 too.
msg258762 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-21 17:39
Semantics are fine (I initially misread what you wanted to do; sorry).

And I wouldn't backport since it's an enhancement and not a bugfix.
msg258792 - (view) Author: Roundup Robot (python-dev) Date: 2016-01-22 11:23
New changeset c873a479a6a3 by Victor Stinner in branch 'default':
site: error on sitecustomize import error
https://hg.python.org/cpython/rev/c873a479a6a3
msg258793 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-01-22 11:23
Brett: "Semantics are fine (I initially misread what you wanted to do; sorry)."

Ok, I pushed my change.

Brett: "And I wouldn't backport since it's an enhancement and not a bugfix."

Ok, fine.
History
Date User Action Args
2016-01-22 11:23:56hayposetstatus: open -> closed
resolution: fixed
messages: + msg258793

versions: - Python 3.5
2016-01-22 11:23:15python-devsetnosy: + python-dev
messages: + msg258792
2016-01-21 17:39:26brett.cannonsetassignee: haypo
2016-01-21 17:39:20brett.cannonsetmessages: + msg258762
2016-01-21 16:59:22hayposetmessages: + msg258758
2016-01-18 09:12:29hayposetmessages: + msg258510
2016-01-13 17:44:42brett.cannonsetmessages: + msg258160
2016-01-13 14:46:56hayposetnosy: + brett.cannon
2016-01-13 14:46:44haypocreate