classification
Title: Improve the error message of addpackage() (site.py) for defective .pth file
Type: behavior Stage: needs patch
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: addpackage in site.py fails hard on badly formed .pth files
View: 5258
Assigned To: Nosy List: Arfrever, dwr2, eric.araujo, r.david.murray, tarek, vstinner
Priority: normal Keywords:

Created on 2010-12-07 03:35 by dwr2, last changed 2010-12-25 04:00 by r.david.murray. This issue is now closed.

Messages (8)
msg123519 - (view) Author: Donald Wallace Rouse II (dwr2) Date: 2010-12-07 03:35
Python 2.7 programs crash on startup due to a defective third-party package installed in site-packages.

starting python 2.7 yields the following error message:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site.py", line 554, in <module>
    main()
  File "/usr/lib64/python2.7/site.py", line 537, in main
    known_paths = addsitepackages(known_paths)
  File "/usr/lib64/python2.7/site.py", line 316, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/usr/lib64/python2.7/site.py", line 192, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/usr/lib64/python2.7/site.py", line 162, in addpackage
    exec line
  File "<string>", line 1, in <module>
KeyError: 'zope'

A similar message appears when starting python2.7 interactively, but it then proceeds to the prompt (doesn't crash).

In the file .../python2.7/site.py, at about line 162, in the function "addpackage", you have an
unprotected "exec" line:
    if line.startswith(("import ", "import\t")):
        exec line

If the execution of the line fails, python generates an uncaught exception.
This places python at the mercy of bugs in third-party software.

The "exec" line should be bracketed by "try/except" to catch such errors:
    if line.startswith(("import ", "import\t")):
        try:
            exec line
        except:
            pass
        continue

Note 1: I am not sure whether this is a Distutils bug or Distutils2 bug (or even something else like "Extension Modules"), so I'm filing it under Distutils2. If this is incorrect, please forward to the proper place. Thanks.

Note 2: Here is where I initially reported this problem: http://bugs.gentoo.org/show_bug.cgi?id=347565
msg123549 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-07 13:50
If the problem is in site.py it doesn't sound like a distutils bug at all.  I remember Victor doing something that changed the error handling/reporting when importing site.py, so I'm adding him as nosy.  I thought that was only in 3.x, though.  Does this not fail in 2.6?
msg123769 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-12-11 02:50
Yes, I patched the C code to not clear exceptions anymore at startup: r78826 (issue #3137). But this issue is different: here the bug is in the 3rd party module (loaded by site.py), not in Python, and Donald proposes to *ignore* errors (where I did the opposite).

> This places python at the mercy of bugs in third-party software.

Why do you consider this as a bug? Yes, Python allows to be extended by 3rd party softwares. And if you install bogus extensions, you break your Python setup.

Ignore errors is never a good idea. How would you notice that the extension is disabled because it is bogus? I prefer explicit errors to force the user to fix its setup.

Extract of the Gentoo issue:

<< The direct cause of your problem is that you manually created
/usr/lib64/python2.7/site-packages/zope/__init__.py file. Please remove it: rm -f /usr/lib64/python2.7/site-packages/zope/__init__.py* >>

I close this issue because I think that ignore errors is not a good idea.

If you disagree, you can still comment this issue, or reopen it.
msg123789 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-12-11 13:46
I suggest to:
- Print path to the .pth file, which causes exception. Current traceback doesn't help in finding the cause of problem:

# echo "import nonexistent" > /usr/lib64/python3.2/site-packages/some_file.pth
# python3.2 -c pass
Traceback (most recent call last):
  File "/usr/lib64/python3.2/site.py", line 520, in <module>
    main()
  File "/usr/lib64/python3.2/site.py", line 509, in main
    known_paths = addsitepackages(known_paths)
  File "/usr/lib64/python3.2/site.py", line 301, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/usr/lib64/python3.2/site.py", line 177, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/usr/lib64/python3.2/site.py", line 148, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
ImportError: No module named nonexistent

- Maybe change error into warning.
msg123793 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-11 14:12
I like the suggestion of turning it into a warning, myself, but you are right that at the least the error message should be improved.
msg123798 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-11 16:01
Aren’t there studies that show that people don’t read warnings?  I’m +0 on a warning and +1 on an error.
msg123814 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-11 19:51
My guess is people don't read warnings when they are a common occurrence.  A working Python should not emit any warnings, and a properly working Python program (post 2.6/3.1 (or whenever it was we decided to suppress deprecation warnings by default)) should not either.  But certainly messages that don't cause program termination *can* be ignored, and thus are more often than program-terminating errors :)

On the other hand, this *is* an error.  If we agree that python startup and site.py should not fail in the face of misconfigured libraries (and we aren't necessarily agreed on that :) then another option would be to use the logging facility to generate an error that would, by default, be logged to stderr but still allow Python to start.  That's not that much different from emitting a warning, functionally, but by having the message make it clear that it is an error it might make it more likely the user would take action.

As for whether or not we should want Python to be able to start up in the face of 3rd party library misconfiguration, I think there are arguments on both sides.  The most compelling argument I can think of for having errors in third partly libraries not cause startup failure is that a user borking their system python by installing a malfunctioning library would then cause all python-dependent system functions to fail to run until they'd fixed the install.  With a system such as gentoo where the package manager that the user might want to use to do the uninstall uses Python, this could be a problem.
msg124629 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-25 04:00
Georg posted a patch to issue 5258 that would "fix" this.  I've posted a counter proposal that would give more info.  We're proposing to simply write to stderr and then continue.  With either patch this issue would be fixed, so I'm making that issue a superseder for this one.
History
Date User Action Args
2010-12-25 04:00:21r.david.murraysetstatus: open -> closed
superseder: addpackage in site.py fails hard on badly formed .pth files
messages: + msg124629

nosy: dwr2, vstinner, tarek, eric.araujo, Arfrever, r.david.murray
resolution: duplicate
2010-12-11 19:51:24r.david.murraysetmessages: + msg123814
2010-12-11 16:49:04vstinnersettitle: site.py crashes on python startup due to defective .pth file -> Improve the error message of addpackage() (site.py) for defective .pth file
2010-12-11 16:01:41eric.araujosetmessages: + msg123798
2010-12-11 14:12:20r.david.murraysetstatus: closed -> open
resolution: not a bug -> (no value)
messages: + msg123793

stage: resolved -> needs patch
2010-12-11 13:46:01Arfreversetmessages: + msg123789
2010-12-11 04:06:43r.david.murraysetstatus: open -> closed
resolution: not a bug
stage: resolved
2010-12-11 02:50:35vstinnersetmessages: + msg123769
2010-12-07 13:50:09r.david.murraysetnosy: + vstinner, r.david.murray
messages: + msg123549
2010-12-07 13:31:46Arfreversetnosy: + Arfrever
2010-12-07 12:48:18eric.araujosetassignee: tarek ->
type: crash -> behavior
components: + Library (Lib), - Distutils2
versions: + Python 3.1, Python 3.2
2010-12-07 03:35:16dwr2create