classification
Title: test___all__ alters utf8 locale setting
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: martin.panter, python-dev, xdegaye
Priority: normal Keywords: patch

Created on 2017-01-14 07:25 by martin.panter, last changed 2017-01-24 06:49 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
readline-getlocale.patch martin.panter, 2017-01-14 07:25 review
Messages (3)
msg285467 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-14 07:25
On a Linux computer I have LANG=en_US.utf8 set. Python 3 initializes the locale with this setting (see revision 43e32b2b4004):

>>> locale.setlocale(locale.LC_CTYPE)  # Get setting initialized by Python
'en_US.utf8'

In Lib/test/test___all__.py, there is a special case to restore the locale after loading the “readline” module (last changed in revision 37ed6eed0595). It does this by calling getlocale() and later setlocale(). However the getlocale() call translates the setting to UTF-8 in uppercase with a dash, and setlocale() keeps it that way:

>>> locale.getlocale(locale.LC_CTYPE)
('en_US', 'UTF-8')
>>> locale.setlocale(locale.LC_CTYPE, _)
'en_US.UTF-8'

This means that the test framework may complain:

Warning -- locale was modified by test___all__

One subtlety is that I only see the warning when the “tkinter” module is unavailable. Otherwise, the test imports Lib/idlelib/iomenu.py, which calls setlocale(locale.LC_CTYPE, "") and restores the original “utf8” spelling.

The easiest fix would be to use setlocale(locale.LC_CTYPE) instead of getlocale() in test___all__. However I think it may be safe to remove this code entirely. The original version of the it was added by Guido in revision ba522c74a931 (year 2002). Three hours later is revision c7d5801bbae8, where Guido changed the “readline” module to restore the locale after Readline library initialization. Since the module is fixed, I don’t think we need the test workaround, and I propose to remove it in Python 3, where it causes the above conflict.
msg285953 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-01-21 12:00
The rationale supporting this change is sound and the patch LGTM.
msg286135 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-01-24 04:57
New changeset 154757703c98 by Martin Panter in branch '3.5':
Issue #29273: Remove unneeded workaround to restore locale
https://hg.python.org/cpython/rev/154757703c98

New changeset 146f7ec82f93 by Martin Panter in branch '3.6':
Issues #29273: Merge test___all__ from 3.5
https://hg.python.org/cpython/rev/146f7ec82f93

New changeset 1615898492a6 by Martin Panter in branch 'default':
Issues #29273: Merge test___all__ from 3.6
https://hg.python.org/cpython/rev/1615898492a6
History
Date User Action Args
2017-01-24 06:49:54martin.pantersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-01-24 04:57:15python-devsetnosy: + python-dev
messages: + msg286135
2017-01-21 12:00:25xdegayesetnosy: + xdegaye
messages: + msg285953
2017-01-14 07:25:54martin.pantercreate