classification
Title: Broken encoding modules are silently skipped.
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: eric.snow, lemburg, python-dev, steve.dower, syeberman
Priority: normal Keywords:

Created on 2016-09-07 18:17 by eric.snow, last changed 2016-09-09 15:59 by steve.dower. This issue is now closed.

Messages (7)
msg274856 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2016-09-07 18:17
In Lib/encodings/__init__.py, search_function() looks up an encoding by name.  It loads the encoding by importing the corresponding module in the encodings package.  If there's an ImportError while importing that module then the encoding gets silently ignored.  I noticed this while working on issue #16384.

While I don't have any direct interest in the matter, I expect that it would be more useful to users if we issued a warning or some other printed message when an encoding is ignored.
msg274933 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-08 00:27
New changeset 2b2e7b39bb68 by Steve Dower in branch 'default':
Issue #28005: Allow ImportErrors in encoding implementation to propagate.
https://hg.python.org/cpython/rev/2b2e7b39bb68
msg274978 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-09-08 06:41
Steve, this is not a correct patch. The purpose of skipping ImportErrors is to give other search functions a chance to find and provide a working codec. Your patch causes this to break.
msg274983 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-09-08 07:35
Thinking about this some, we could go ahead and document somewhere that if a codec wants to tell the search function that it cannot work due to some missing dependency, it will have to raise an ImportError from the codec module level (instead of just lettering ImportError bubble up), possibly reraising the error to assure this.
msg275011 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-09-08 13:18
Is it deliberate that inner import errors should be treated as if the codec is missing? Another search function isn't going to help in that case unless it finds the same encoding in another place. I didn't see any evidence of that, and apparently neither did Eric (but I was bitten by a misspelt import within my new encodings/oem.py giving a confusing message).

I don't think any encoding modules in the stdlib should fail to import. This change ensures that we hear about it if they do fail.
msg275279 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-09-09 07:30
Yes, that's intended. See e.g. the mbcs codec.

If a search function gets an ImportError, it interprets this as "codec is not available/working", and then gives other search functions a chance to find a working one.
msg275325 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-09-09 15:59
I reverted the change but wrote the wrong issue number. Also added a comment so the next person to encounter this realises it is the correct behaviour.

New changeset 4e80a157ea66 by Steve Dower in branch 'default':
Revert #27959: ImportError within an encoding module should also skip the encoding
https://hg.python.org/cpython/rev/4e80a157ea66
History
Date User Action Args
2016-09-09 15:59:19steve.dowersetstatus: open -> closed
resolution: not a bug
messages: + msg275325
2016-09-09 07:30:17lemburgsetmessages: + msg275279
2016-09-08 13:18:20steve.dowersetmessages: + msg275011
2016-09-08 07:35:38lemburgsetmessages: + msg274983
2016-09-08 06:41:03lemburgsetstatus: closed -> open

nosy: + lemburg
messages: + msg274978

resolution: fixed -> (no value)
2016-09-08 00:28:21steve.dowersetstatus: open -> closed

nosy: + steve.dower
assignee: steve.dower
resolution: fixed
stage: test needed -> resolved
2016-09-08 00:27:51python-devsetnosy: + python-dev
messages: + msg274933
2016-09-07 20:59:07syebermansetnosy: + syeberman
2016-09-07 18:17:08eric.snowcreate