classification
Title: Empty PYTHONIOENCODING is not the same as nonexistent
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, haypo, ncoghlan, python-dev, serhiy.storchaka
Priority: normal Keywords: needs review, patch

Created on 2013-08-23 06:45 by serhiy.storchaka, last changed 2014-02-25 15:55 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
empty_pythonioencoding.patch serhiy.storchaka, 2013-08-23 06:55 review
empty_pythonioencoding_2.patch serhiy.storchaka, 2013-08-23 08:04 review
empty_pythonioencoding_3.patch serhiy.storchaka, 2013-08-23 19:11 review
Messages (9)
msg195946 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-08-23 06:45
$ PYTHONIOENCODING= ./python
Fatal Python error: Py_Initialize: can't initialize sys standard streams
LookupError: unknown encoding: 
Aborted (core dumped)

As a consequence we can't set only the error handler.

$ PYTHONIOENCODING=:surrogateescape ./python
Fatal Python error: Py_Initialize: can't initialize sys standard streams
LookupError: unknown encoding: 
Aborted (core dumped)

Here is a patch which allows accepting empty encoding as default encoding.
msg195947 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-08-23 07:44
Patch looks good to me.
msg195950 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-08-23 08:01
Patch looks good to me, but you have to update Doc/using/cmdline.rst,
at least to add a versionchanged section.

Tests would also be nice :-)

I really like the the "PYTHONIOENCODING=:surrogateescape" use case!
msg195951 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-08-23 08:04
Second variant of the patch also supports empty error handler as the default error handler (i.e. strict).

$ PYTHONIOENCODING=ascii: ./python
Python 3.4.0a1+ (default:5b5ef012cd4e+, Aug 23 2013, 10:18:51) 
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print('\xbd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character '\xbd' in position 0: ordinal not in range(128)

Without it different error is raised:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
LookupError: unknown error handler name ''
msg196007 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-08-23 19:11
Here is a patch with tests and documentation. For the documentation I hope on your help.
msg197501 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-09-11 18:29
Nick, could you help me with documentation? I'm sure my wording needs improving.

And there is one question left. Is this change appropriate for 2.7 and 3.3, or only for 3.4?
msg197560 - (view) Author: Roundup Robot (python-dev) Date: 2013-09-13 08:46
New changeset c7fdb0637d0b by Serhiy Storchaka in branch 'default':
Issue #18818: The "encodingname" part of PYTHONIOENCODING is now optional.
http://hg.python.org/cpython/rev/c7fdb0637d0b
msg197561 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-09-13 08:48
Thank you Ezio and Vajrasky for the review.
msg212186 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-25 15:55
New changeset 8ac9c3754d33 by R David Murray in branch 'default':
whatsnew: encoding is now optional in PYTHONIOENCODING (#18818)
http://hg.python.org/cpython/rev/8ac9c3754d33
History
Date User Action Args
2014-02-25 15:55:52python-devsetmessages: + msg212186
2013-09-13 08:48:41serhiy.storchakasetstatus: open -> closed
versions: - Python 2.7, Python 3.3
type: behavior -> enhancement
messages: + msg197561

resolution: fixed
stage: patch review -> resolved
2013-09-13 08:46:52python-devsetnosy: + python-dev
messages: + msg197560
2013-09-11 18:29:49serhiy.storchakasetmessages: + msg197501
2013-09-02 15:41:48serhiy.storchakasetkeywords: + needs review
2013-08-23 19:11:14serhiy.storchakasetfiles: + empty_pythonioencoding_3.patch

messages: + msg196007
2013-08-23 09:10:00Arfreversetnosy: + Arfrever
2013-08-23 08:04:34serhiy.storchakasetfiles: + empty_pythonioencoding_2.patch

messages: + msg195951
2013-08-23 08:01:17hayposetmessages: + msg195950
2013-08-23 07:44:43ncoghlansetmessages: + msg195947
2013-08-23 06:55:05serhiy.storchakasetfiles: + empty_pythonioencoding.patch
2013-08-23 06:54:45serhiy.storchakasetfiles: - empty_pythonioencoding.patch
2013-08-23 06:46:35serhiy.storchakalinkissue18713 dependencies
2013-08-23 06:45:48serhiy.storchakacreate