classification
Title: setlocale, getlocale succession --> ValueError or (None, None)
Type: behavior Stage: resolved
Components: macOS Versions: Python 3.3, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: locale.getdefaultlocale() fails on Mac OS X with default language set to English
View: 18378
Assigned To: ronaldoussoren Nosy List: albertjan, ned.deily, r.david.murray, ronaldoussoren
Priority: normal Keywords:

Created on 2014-03-20 18:50 by albertjan, last changed 2014-03-22 21:27 by r.david.murray. This issue is now closed.

Messages (7)
msg214260 - (view) Author: albertjan (albertjan) Date: 2014-03-20 18:50
------------->> see also issue #18378

# Result applies to Python 2.7.2 and Python 3.3.4
# Mac OS X Mountain Lion 10.9.1 on Virtualbox with a Linux Debian AMD-64 host
fomcls-Mac-Pro:~ fomcl$ uname -a
Darwin fomcls-Mac-Pro.local 12.2.0 Darwin Kernel Version 12.2.0: Sat Aug 25 00:48:52 PDT 2012; root:xnu-2050.18.24~1/RELEASE_X86_64 x86_6

>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'C/UTF-8/C/C/C/C'
>>> locale.getlocale()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 515, in getlocale
    return _parse_localename(localename)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 428, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

########################################################################
# below another configuration (no hackintosh)
########################################################################


conda 2.7:

Python 2.7.6 |Continuum Analytics, Inc.| (default, Jan 10 2014, 11:23:15)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'C'
>>> locale.getlocale()
(None, None)

conda 3.3:

Python 3.3.5 |Continuum Analytics, Inc.| (default, Mar 10 2014, 11:22:25)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'C'
>>> locale.getlocale()
(None, None)

Regular 2.7:

Python 2.7.5 (default, Aug 25 2013, 00:04:04)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'C'
>>> locale.getlocale()
(None, None)
>>>


Regular 3.3  (broken installation??)
Python 3.3.2 (v3.3.2:d047928ae3f6, May 13 2013, 13:52:24)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
Segmentation fault: 11


########################################################################
### finally, the expected result (on Linux)
########################################################################

antonia@antonia-HP-2133 ~ $ uname -a
Linux antonia-HP-2133 3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:32:08 UTC 2012 i686 i686 i686 GNU/Linux
Python 2.7.3 (default, Feb 27 2014, 19:39:10) [GCC 4.7.2] on linux2
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'en_US.UTF-8'
>>> locale.getlocale()
('en_US', 'UTF-8')
msg214264 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-20 19:28
Have you tried experimenting with what locale is set to?  These look like locale configuration issues rather than platform issues.  Specifically, the C locale will produce the (None, None) result on linux.
msg214329 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-03-21 07:12
It's hard to be absolutely sure what is going on here since you show several different interpreters and appear to be running on a non-standard, unsupported platform but, as David noted, the primary issue is that the process locale has almost certainly not been set to a correct value.  In general, OS X does not set a default locale for a login process, although programs like Terminal.app may do so by default when they start a login shell for you.  But if you login via ssh, for example, no locale environment variable will be set unless you do it yourself.  You can examine the environment variables in the running interpreter with:

>>> import os,pprint
>>> pprint.pprint(dict(os.environ))
[...]
'LANG': 'en_US.UTF-8',
[...]

Look for LANG or LC_ALL variables.  You can also use locale.getdefaultlocale() to see what the locale module is using as a default.  You can override or establish a default by setting the LANG environment variable in your shell session; to do it "permanently", add the setting to your login shell's startup profile.  For example:

export LANG=en_US.UTF-8

The secondary issue is the segfault.  You say you are running on "Mountain Lion 10.9.1" but OS X Mountain Lion is 10.8.x.  If you are, in fact, running on 10.9.1 (also known an Mavericks) with the python.org 3.3.2, the segfault using the interactive interpreter is the problem documented in Issue18458 and has nothing to do with locale.  The solution for that is to upgrade to a more recent Python 3, either Python 3.3.5 or the newly-released 3.4.0.
msg214331 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2014-03-21 08:10
The locale issue is that on a default (us english) install of 10.9 the following locale related environment variables are set:

$ set | grep UTF
LANG=en_US.UTF-8
LC_CTYPE=UTF-8

The locale module doesn't understand the LC_CTYPE setting, and this appears to be a at best under documented feature of OSX: you can use "UTF-8" as the locale name for LC_CTYPE and this is handled sanely by libc. 

Issue #18378 appears to be related to this.

IMHO this is a real bug and should be fixed because the locale module's documentation says it exposes the C library's functionality.
msg214398 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-03-21 18:33
OK, let's close this issue as a duplicate of Issue18378 and continue the discussion there.
msg214515 - (view) Author: albertjan (albertjan) Date: 2014-03-22 20:34
Ok, I know this is closed as a duplicate, but I am pasting some additional info here for reference. All info is about the FIRST system of the original message

## The locale settings
fomcls-Mac-Pro:Desktop fomcl$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

##  export LANG="en_US.UTF-8" does not fix it.
fomcls-Mac-Pro:Desktop fomcl$ export LANG="en_US.UTF-8"
fomcl-Mac-Pro:Desktop fomcl$ python
Python 2.7.2 (default, Jun 20 2012, 16:23:33) 
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
>>> import locale
>>> locale.getdefaultlocale()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 496, in getdefaultlocale
    return _parse_localename(localename)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 428, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8
>>> locale.setlocale(locale.LC_ALL, "")
'en_US.UTF-8/UTF-8/en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/en_US.UTF-8'
>>> locale.getlocale()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 515, in getlocale
    return _parse_localename(localename)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 428, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

## export LC_ALL="en_US.UTF-8" makes it all work as expected/desired.
fomcls-Mac-Pro:Desktop fomcl$ export LC_ALL="en_US.UTF-8"
fomcls-Mac-Pro:Desktop fomcl$ python
Python 2.7.2 (default, Jun 20 2012, 16:23:33) 
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
>>> import locale
>>> locale.getdefaultlocale()
('en_US', 'UTF-8')
>>> locale.getlocale()
(None, None)
>>> locale.setlocale(locale.LC_ALL, "")
'en_US.UTF-8'
>>> locale.getlocale()
('en_US', 'UTF-8')
msg214519 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-22 21:27
Thank you, that information confirms that this is indeed a duplicate.
History
Date User Action Args
2014-03-22 21:27:41r.david.murraysetmessages: + msg214519
2014-03-22 20:34:42albertjansetmessages: + msg214515
2014-03-21 18:33:44ned.deilysetstatus: open -> closed
superseder: locale.getdefaultlocale() fails on Mac OS X with default language set to English
resolution: works for me -> duplicate
messages: + msg214398
2014-03-21 08:10:46ronaldoussorensetstatus: pending -> open

messages: + msg214331
2014-03-21 07:12:02ned.deilysetstatus: open -> pending

nosy: + ned.deily
messages: + msg214329

resolution: works for me
stage: resolved
2014-03-20 19:28:36r.david.murraysetnosy: + r.david.murray
messages: + msg214264
2014-03-20 18:50:08albertjancreate