Author benjamin.peterson
Recipients amaury.forgeotdarc, benjamin.peterson, brett.cannon, sjmachin, vstinner
Date 2009-01-30.02:00:51
SpamBayes Score 2.42742e-09
Marked as misclassified No
Message-id <>
In-reply-to <>
On Thu, Jan 29, 2009 at 5:13 PM, STINNER Victor <> wrote:
> Ok, I created a new function PyTokenizer_FromUnicode(). I
> choosed "FromUnicode" because the string is encoded in unicode (as
> UTF-8, even if it's not the wchar_t* type).

How about PyTokenizer_FromUTF8() then?

>> The (char *) cast in PyTokenizer_FromString is unneeded.
> The cast on the decode_str() result? It was already present in the
> original code. I removed it in my new patch.

No, I was referring to this line:

tok->encoding = (char *)PyMem_MALLOC
Date User Action Args
2009-01-30 02:00:54benjamin.petersonsetrecipients: + benjamin.peterson, brett.cannon, sjmachin, amaury.forgeotdarc, vstinner
2009-01-30 02:00:52benjamin.petersonlinkissue4626 messages
2009-01-30 02:00:51benjamin.petersoncreate