classification
Title: _elementtree: Raise ImportError when importing of pyexpat fails
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eli.bendersky Nosy List: Arfrever, benjamin.peterson, eli.bendersky, flox, georg.brandl, jerub, python-dev, serhiy.storchaka, zmedico
Priority: low Keywords: patch

Created on 2012-06-03 07:21 by Arfrever, last changed 2015-11-26 17:18 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
elementtree_importerror.patch jerub, 2012-07-07 21:03 review
Messages (11)
msg162198 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2012-06-03 07:21
If, after building of Python, libexpat.so (library used by pyexpat module) has been broken/removed or pyexpat module has been broken/removed, then attempt of import of _elementtree module, which requires pyexpat module, will raise strange exceptions in Python 3.

These exceptions can be simulated by setting sys.modules["pyexpat"] = None.

$ python2.6 -c 'import sys; sys.modules["pyexpat"] = None; import _elementtree'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named pyexpat
$ python2.7 -c 'import sys; sys.modules["pyexpat"] = None; import _elementtree'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: PyCapsule_Import could not import module "pyexpat"
$ python3.1 -c 'import sys; sys.modules["pyexpat"] = None; import _elementtree'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SystemError: initialization of _elementtree raised unreported exception
$ python3.2 -c 'import sys; sys.modules["pyexpat"] = None; import _elementtree'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SystemError: initialization of _elementtree raised unreported exception
$ python3.3 -c 'import sys; sys.modules["pyexpat"] = None; import _elementtree'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1286, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1253, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 432, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 347, in set_package_wrapper
  File "<frozen importlib._bootstrap>", line 360, in set_loader_wrapper
  File "<frozen importlib._bootstrap>", line 870, in load_module
RuntimeError: cannot load dispatch table from pyexpat

I suggest to raise ImportError instead of SystemError or RuntimeError.
Third-party modules would rather use only 'except ImportError: pass' instead of 'except (ImportError, SystemError, RuntimeError): pass' when trying to optionally use e.g. xml.etree.cElementTree.
xml.etree.ElementTree in Python 3.3 also ignores only ImportError when trying to import _elementtree.
msg164937 - (view) Author: Stephen Thorne (jerub) * Date: 2012-07-07 21:03
With the attached patch, with python3.3(trunk) I instead get: 

./python.exe  -c 'import _elementtree'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1294, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1261, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 432, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 347, in set_package_wrapper
  File "<frozen importlib._bootstrap>", line 360, in set_loader_wrapper
  File "<frozen importlib._bootstrap>", line 872, in load_module
ImportError: PyCapsule_Import could not import module "pyexpat"

(I have deleted pyexpat.so out of the build for the purposes of testing)

RuntimeError will continue to be raised in the case the version is wrong.
msg165500 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-07-15 03:45
I agree, but this will now be delayed to 3.4, since it's not a bug-fixing change in functionality.

P.S. is there any way to create a test for it?
msg165543 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2012-07-15 18:34
IMHO wrong exception could be treated as a bug, which could be fixed in 3.2 and 3.3.
Benjamin Peterson and Georg Brandl: What do you think?

> P.S. is there any way to create a test for it?

You can set sys.modules["pyexpat"]=None and test exception type when trying to import _elementtree.
msg165563 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-07-16 01:26
IMHO, it can be fixed as people were relying on the old behavior.
msg165678 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-07-17 04:07
I tried blocking the import of 'pyexpat' in a test by using test.support.import_fresh_module, placing 'pyexpat' into the blocked list, but it doesn't work (i.e. pyexpat, if it exists, is still imported)
msg165679 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-07-17 04:11
Benjamin, what "old behavior"? Did it happen to raise ImportError historically and only recent changes made it RuntimeError? I'm not sure  this is the case.
msg165680 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2012-07-17 05:30
> Did it happen to raise ImportError historically and only recent
> changes made it RuntimeError? I'm not sure  this is the case.

ImportError was raised in Python 2.
msg165708 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-07-17 11:25
New changeset d896fd0a8ba7 by Eli Bendersky in branch '3.2':
ISsue #14988: restore Python 2's behavior of raising ImportError when unable to load pyexpat, instead of a SystemError/RuntimeError
http://hg.python.org/cpython/rev/d896fd0a8ba7

New changeset c8774ff45733 by Eli Bendersky in branch 'default':
Merge for #14988
http://hg.python.org/cpython/rev/c8774ff45733
msg165709 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-07-17 11:26
Committed fixes to raise ImportError in both Python 3.2 and 3.3

At the moment no test added because I'm having trouble using import_fresh_module to avoid re-importing of pyexpat in the etree tests (it gets imported anyway).
msg255059 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-21 15:14
Can this issue be closed?
History
Date User Action Args
2015-11-26 17:18:16serhiy.storchakasetstatus: pending -> closed
resolution: fixed
stage: test needed -> resolved
2015-11-21 15:14:22serhiy.storchakasetstatus: open -> pending
nosy: + serhiy.storchaka
messages: + msg255059

2012-07-17 11:26:47eli.benderskysetmessages: + msg165709
2012-07-17 11:25:42python-devsetnosy: + python-dev
messages: + msg165708
2012-07-17 05:30:05Arfreversetmessages: + msg165680
2012-07-17 04:11:28eli.benderskysetmessages: + msg165679
2012-07-17 04:07:57eli.benderskysetmessages: + msg165678
2012-07-16 01:26:44benjamin.petersonsetmessages: + msg165563
2012-07-15 18:34:15Arfreversetnosy: + georg.brandl, benjamin.peterson
messages: + msg165543
2012-07-15 03:45:24eli.benderskysetpriority: normal -> low
versions: - Python 3.3
messages: + msg165500

assignee: eli.bendersky
type: behavior
2012-07-07 21:03:41jerubsetfiles: + elementtree_importerror.patch

nosy: + jerub
messages: + msg164937

keywords: + patch
2012-07-06 03:13:48eli.benderskysetstage: test needed
components: + Library (Lib)
versions: + Python 3.4, - Python 3.2
2012-06-03 07:26:12zmedicosetnosy: + zmedico
2012-06-03 07:21:55Arfrevercreate