Issue10466
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2010-11-20 11:36 by skoczian, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (17) | |||
---|---|---|---|
msg121616 - (view) | Author: Sibylle Koczian (skoczian) | Date: 2010-11-20 11:36 | |
Running locale.py as a module on Windows, using the Python command window, produces this output: C:\Python31\Lib>locale.py Locale aliasing: Locale defaults as determined by getdefaultlocale(): ------------------------------------------------------------------------ Language: de_DE Encoding: cp1252 Locale settings on startup: ------------------------------------------------------------------------ LC_NUMERIC ... Language: (undefined) Encoding: (undefined) LC_MONETARY ... Language: (undefined) Encoding: (undefined) LC_COLLATE ... Language: (undefined) Encoding: (undefined) LC_CTYPE ... Language: (undefined) Encoding: (undefined) LC_TIME ... Language: (undefined) Encoding: (undefined) Locale settings after calling resetlocale(): ------------------------------------------------------------------------ Traceback (most recent call last): File "C:\Python31\Lib\locale.py", line 1798, in <module> _print_locale() File "C:\Python31\Lib\locale.py", line 1761, in _print_locale resetlocale() File "C:\Python31\Lib\locale.py", line 537, in resetlocale _setlocale(category, _build_localename(getdefaultlocale())) locale.Error: unsupported locale setting Tried with 2.7 on Windows XP, 32bit, 3.1.2 and 3.2a4 on Windows 7, 64bit. System character set in all cases cp1252 as usual for a German windows installation. With 2.6, 2.6.5, 3.1.2 on Linux with UTF-8 system character set: no exception, expected output. |
|||
msg121656 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2010-11-20 14:37 | |
unsupported locale setting is a message that comes from the C runtime, IIUC. Does it work on windows with 2.6? |
|||
msg121907 - (view) | Author: Sibylle Koczian (skoczian) | Date: 2010-11-21 12:06 | |
-----Ursprüngliche Nachricht----- Von: "R. David Murray" <report@bugs.python.org> Gesendet: 20.11.2010 15:37:28 An: nulla.epistola@web.de Betreff: [issue10466] locale.py throws exception on Windows / Non-UTF8 system > >R. David Murray <rdmurray@bitdance.com> added the comment: > >unsupported locale setting is a message that comes from the C runtime, IIUC. Does it work on windows with 2.6? > No, it doesn't. I installed 2.6.6, and I found an old 2.3 (installed by some application). Both of them throw the same exception if locale is run as a script. ___________________________________________________________ GRATIS! Movie-FLAT mit über 300 Videos. Jetzt freischalten unter http://movieflat.web.de |
|||
msg121961 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2010-11-21 19:04 | |
See also issue 1699853. The problem here is that resetlocale calls getdefaultlocale, and getdefaultlocale returns something that setlocale cannot consume on Windows. For example, on my US windows system, getlocale returns ('English_United States', '1252'), while getdefaultlocale returns ('en_US', 'cp1252'). The latter results in the 'unsupported locale settting' error message when passed to setlocale. Since getdefaultlocale contains special-case code for the win32 platform, I wonder if this is a bug in getdefaultlocale. I suspect it is operating as designed, but is that operation useful if the result isn't recognized by the platform in question? Or if, as it appears from looking at the Python code (but I haven't looked at the C code) the data is actually coming from the C runtime, perhaps the bug is that setlocale doesn't accept the output of getdefaultlocale on windows? Hopefully Mark Andre will have an answer, but I've also added our windows experts to nosy to see if they have any insights. Given the long standing nature of this issue, it seems that the priority isn't very high, especially since the bug is revealed not by user code but by a diagnostic routine in the stdlib. |
|||
msg122001 - (view) | Author: Sibylle Koczian (skoczian) | Date: 2010-11-21 21:51 | |
Am 21.11.2010 20:04, schrieb R. David Murray: > > R. David Murray<rdmurray@bitdance.com> added the comment: > > Given the long standing nature of this issue, it seems that the > priority isn't very high, especially since the bug is revealed not by > user code but by a diagnostic routine in the stdlib. > It's true that the bug is quite old, but first it crashed my application, then I found that locale.py doesn't run as a script. I put that into the tracker simply because it's so easy to reproduce. But this is almost as simple (and it's the same bug, isn't it?): Python 3.1.2 (r312:79149, Mar 20 2010, 22:55:39) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import calendar >>> kal = calendar.localeHTMLCalendar() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'module' object has no attribute 'localeHTMLCalendar' >>> kal = calendar.LocaleHTMLCalendar() >>> kal_aus = kal.formatyearpage(2011) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python31\lib\calendar.py", line 478, in formatyearpage a(self.formatyear(theyear, width)) File "C:\Python31\lib\calendar.py", line 454, in formatyear a(self.formatmonth(theyear, m, withyear=False)) File "C:\Python31\lib\calendar.py", line 427, in formatmonth a(self.formatmonthname(theyear, themonth, withyear=withyear)) File "C:\Python31\lib\calendar.py", line 546, in formatmonthname with different_locale(self.locale): File "C:\Python31\lib\calendar.py", line 489, in __enter__ self.oldlocale = _locale.setlocale(_locale.LC_TIME, self.locale) File "C:\Python31\lib\locale.py", line 527, in setlocale return _setlocale(category, locale) locale.Error: unsupported locale setting >>> This would be quite unfunny if I couldn't run my script on Linux instead. Sibylle |
|||
msg122065 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2010-11-22 02:13 | |
It is unfunny that your program fails on Windows. Yes it is the same bug, since calendar calls getdefaultlocale. Issue 1080864 makes very interesting reading in this context. Clearly Martin is right, yet the OP seems to find the current behavior of getdefaultencoding (returning the more standard locale name rather than Windows unique values) useful. It is an interesting question what can be done about this. Clearly resetlocale is non-functional on Windows, and getdefaultlocale is of questionable utility. At the very least these limitations need to be documented. Your application bug, however, can probably be addressed by fixing calendar to not use getdefaultencoding. We should open a new bug for that, though. |
|||
msg122071 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2010-11-22 02:37 | |
I've opened issue 10498 for the calendar bug. |
|||
msg122090 - (view) | Author: Ned Deily (ned.deily) * | Date: 2010-11-22 04:26 | |
FYI, calendar.localeHTMLCalendar() fails on OS X in exactly the same way for exactly the same reason. See, for instance, Issue10090. |
|||
msg122119 - (view) | Author: Marc-Andre Lemburg (lemburg) * | Date: 2010-11-22 13:09 | |
I think that's a bug in the resetlocale() API. The correct way to reset the locale setting to defaults, it to use setlocale(category, "") The other issues here is that getlocale() appears to return non-ISO language codes on Windows. If that's indeed the case, then we would need to add mappings of the Windows codes to the ISO ones and use a reverse mappings to make setlocale() work with the ISO codes. Perhaps it's easier to just update the mapping used by getdefaultencoding() to return the non-ISO codes used by Windows and then update the documentation to say that on Windows, non-ISO codes are returned. BTW: I wonder why this hasn't popped up earlier. |
|||
msg122127 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2010-11-22 14:22 | |
I had a report from a user on IRC during the bug weekend that they could not reproduce the failure on windows. So it may be dependent on the windows version. That doesn't answer your question of why it hasn't come up before, though, since my tests were done on XP. |
|||
msg122216 - (view) | Author: Marc-Andre Lemburg (lemburg) * | Date: 2010-11-23 14:15 | |
R. David Murray wrote: > > R. David Murray <rdmurray@bitdance.com> added the comment: > > I had a report from a user on IRC during the bug weekend that they could not reproduce the failure on windows. So it may be dependent on the windows version. That doesn't answer your question of why it hasn't come up before, though, since my tests were done on XP. Some research shows that the MS VCRT uses non-ISO locale names for setlocale(): http://msdn.microsoft.com/en-us/library/x99tb11d.aspx http://msdn.microsoft.com/en-us/library/hzz3tw78.aspx http://msdn.microsoft.com/en-us/library/39cwe7zf.aspx http://msdn.microsoft.com/en-us/library/cdax410z.aspx and it doesn't support the ISO style locale namings, even though the setlocale() page says "The set of available languages, country/region codes, and code pages includes all those supported by the Win32 NLS API" and the Win32 API does support the ISO names: http://msdn.microsoft.com/en-us/library/dd373814(VS.85).aspx I'll have to check whether Vista also shows this behavior. On Win XP the setlocale() API doesn't accept ISO language names; I can confirm that. It only accepts the fully written out form described on the above pages. |
|||
msg122217 - (view) | Author: Marc-Andre Lemburg (lemburg) * | Date: 2010-11-23 14:50 | |
Marc-Andre Lemburg wrote: > > Marc-Andre Lemburg <mal@egenix.com> added the comment: > > R. David Murray wrote: >> >> R. David Murray <rdmurray@bitdance.com> added the comment: >> >> I had a report from a user on IRC during the bug weekend that they could not reproduce the failure on windows. So it may be dependent on the windows version. That doesn't answer your question of why it hasn't come up before, though, since my tests were done on XP. > > Some research shows that the MS VCRT uses non-ISO locale names > for setlocale(): > > http://msdn.microsoft.com/en-us/library/x99tb11d.aspx > http://msdn.microsoft.com/en-us/library/hzz3tw78.aspx > http://msdn.microsoft.com/en-us/library/39cwe7zf.aspx > http://msdn.microsoft.com/en-us/library/cdax410z.aspx > > and it doesn't support the ISO style locale namings, even though > the setlocale() page says "The set of available languages, country/region codes, and code pages > includes all those supported by the Win32 NLS API" and the Win32 > API does support the ISO names: > > http://msdn.microsoft.com/en-us/library/dd373814(VS.85).aspx > > I'll have to check whether Vista also shows this behavior. On > Win XP the setlocale() API doesn't accept ISO language names; > I can confirm that. It only accepts the fully written out > form described on the above pages. Confirmed on Vista as well. I think the only choice we have is to add a new alias dictionary mapping the ISO language names to the Windows ones. |
|||
msg136036 - (view) | Author: Tim Lyons (guy.linton) | Date: 2011-05-15 16:52 | |
Surely the user of getlocale as the right to expect that the same thing (i.e. ISO language codes) would be returned on all platforms. I am looking at some code that provides special purpose date handling routines. The appropriate routine is selected by a language code (for the locale category LC_TIME as it happens) like fr_FR. In order to get this to work on different platforms, you have to test whether it is windows, and if so use getdefaultlocale, while for other platforms, you use getlocale in order to return similar language codes. This code would actually be wrong if the application were to change the locale inside the application, because on other platforms the change would have an effect, while on windows, it would be ignored! [I am running on Mac OS X as it happens, not Linux or Windows]. So, don’t say that getdefaultloale is not useful, it is the one that is needed on Windows! Note this is not a problem with the encoding – some of the discussion and many of the related bugs are concerned with the encoding. Also note Marc-Andre pointing out in http://mail.python.org/pipermail/python-bugs-list/2004-December/026667.html that “getdefaultlocale() mimics the lookup mechanism of setlocale(LC_ALL, "")”. I recognise that he is talking about the lookup mechanism rather than the results, but it seems to suggest that the results might be similar (if you do the same lookup you would get the same result). But “getdefaultlocale returns something that setlocale cannot consume on Windows”! I recognise that it would be difficult to decide to code locale on python so that windows returns something that is not the same as the native call, but it does seem to me that python locale should translate the Windows language codes so that they are the same as are returned on all the other platforms (i.e. ISO codes etc). |
|||
msg220349 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2014-06-12 13:28 | |
See issue 21731 for considering putting a workaround for this into the calendar module (noted here because of msg122065). |
|||
msg220350 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2014-06-12 13:33 | |
Oh, I see I'd already previously opened issue 10498 for that. |
|||
msg350303 - (view) | Author: Vidar Fauske (vidartf) * | Date: 2019-08-23 15:03 | |
Note that this is still an issue on Windows: >>> import locale >>> locale.getdefaultlocale() ('en_US', 'cp1252') >>> locale.setlocale(locale.getdefaultlocale()) Error: unsupported locale setting |
|||
msg351786 - (view) | Author: Steve Dower (steve.dower) * | Date: 2019-09-11 09:49 | |
issue37945 is a duplicate and has more up-to-date information on it, so I'm going to close this one even though it's older. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:57:09 | admin | set | github: 54675 |
2021-12-06 23:44:20 | iritkatriel | link | issue21731 superseder |
2019-09-11 09:49:22 | steve.dower | set | status: open -> closed superseder: [Windows] test_locale.TestMiscellaneous.test_getsetlocale_issue1813() fails messages: + msg351786 resolution: duplicate stage: resolved |
2019-08-23 15:03:50 | vidartf | set | versions:
+ Python 3.7, - Python 3.1, Python 3.2 nosy: + paul.moore, vidartf, zach.ware, steve.dower messages: + msg350303 components: + Windows |
2015-08-25 13:15:50 | Gabi.Davar | set | nosy:
+ Gabi.Davar |
2014-06-12 13:33:31 | r.david.murray | set | messages: + msg220350 |
2014-06-12 13:28:31 | r.david.murray | set | messages: + msg220349 |
2011-05-15 16:52:23 | guy.linton | set | nosy:
+ guy.linton messages: + msg136036 |
2011-05-07 07:33:28 | iki | set | nosy:
+ iki |
2010-11-23 14:50:02 | lemburg | set | messages: + msg122217 |
2010-11-23 14:15:19 | lemburg | set | messages: + msg122216 |
2010-11-22 14:22:19 | r.david.murray | set | messages: + msg122127 |
2010-11-22 13:09:13 | lemburg | set | messages: + msg122119 |
2010-11-22 04:26:23 | ned.deily | set | nosy:
+ ned.deily messages: + msg122090 |
2010-11-22 02:37:41 | r.david.murray | set | messages: + msg122071 |
2010-11-22 02:13:19 | r.david.murray | set | nosy:
+ loewis messages: + msg122065 title: locale.py resetlocale throws exception on Windows -> locale.py resetlocale throws exception on Windows (getdefaultlocale returns value not usable in setlocale) |
2010-11-21 21:51:54 | skoczian | set | messages: + msg122001 |
2010-11-21 19:04:41 | r.david.murray | set | priority: normal -> low title: locale.py throws exception on Windows / Non-UTF8 system -> locale.py resetlocale throws exception on Windows nosy: + tim.golden, brian.curtin messages: + msg121961 type: behavior |
2010-11-21 12:06:28 | skoczian | set | messages: + msg121907 |
2010-11-20 14:37:27 | r.david.murray | set | nosy:
+ r.david.murray, lemburg messages: + msg121656 |
2010-11-20 11:36:05 | skoczian | create |