This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients Arfrever, lemburg, loewis, pitrou, vstinner
Date 2010-05-07.10:54:02
SpamBayes Score 2.837706e-06
Marked as misclassified No
Message-id <201005071253.57122.victor.stinner@haypocalc.com>
In-reply-to <4BE3DB33.9090904@egenix.com>
Content
Le vendredi 07 mai 2010 11:19:52, vous avez écrit :
> > Ok, you conviced me with your PYTHONFSENCODING suggestion (#8622). Can
> > you review my last patch please?
> 
> I don't think we can change the fallback encoding in 3.2. But you
> can start a discussion on python-dev, of course.

Ok, I will ask on python-dev.

> +		fprintf(stderr,
> +			"Unable to get the locale encoding: "
> +			"fallback to utf-8\n");
> 
> This would have to read "... to ASCII"

Fixed.

> 
> +		Py_FileSystemDefaultEncoding = "ascii";
> 
> +	codec = _PyCodec_Lookup(Py_FileSystemDefaultEncoding);
> +	if (!codec) {
> +		Py_FatalError(
> +		    "Py_Initialize: unable to load the file system codec");
> 
> It's better to use the same approach as above for this situation
> as well.

I choosed to display a fatal error here to give a more revelant error message 
to the user. Without the check, _PyCodec_Lookup() will fail anyway, but later 
in a random function :-/

The fatal error only occurs in critical situations: no more memory, import 
machinery completly broken (eg. #8611), etc. In this case, fallback to ASCII 
doesn't help, it will also raise somewhere later.

About nl_langinfo(CODESET): get_codeset() does already reject unknown 
encoding. So this call is only done on known encoding names.

> You also need to change this line in pythonrun.c:
> 
>         /* reset file system default encoding */
>         if (!Py_HasFileSystemDefaultEncoding) {
>                 free((char*)Py_FileSystemDefaultEncoding);
>                 Py_FileSystemDefaultEncoding = NULL;
>         }

Fixed. This test only match if get_codeset() is used: I choosed to set the 
encoding to ASCII with Py_HasFileSystemDefaultEncoding=0.

> I'm not sure what the purpose of Py_HasFileSystemDefaultEncoding
> is.

Its name doesn't help. It's just a flag to tell if free() should be called or 
not... (see _Py_SetFileSystemEncoding()).

> In any case, initfsencoding() would always have to set that
> flag to 1.

initfsencoding() is a static function and it's only called by 
Py_InitializeEx(). Can Py_InitializeEx() be called multiple times?
History
Date User Action Args
2010-05-07 10:54:07vstinnersetrecipients: + vstinner, lemburg, loewis, pitrou, Arfrever
2010-05-07 10:54:04vstinnerlinkissue8610 messages
2010-05-07 10:54:02vstinnercreate