This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients serhiy.storchaka, vladistan, vstinner
Date 2013-04-14.01:38:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1365903535.72.0.0813124801856.issue17694@psf.upfronthosting.co.za>
In-reply-to
Content
Attached patch changes _PyUnicodeWriter_Init() API: it now only has one argument (the writer). Minimum length and overallocation must be configured using attributes. The problem with the old API was that it was not possible to configure minimum length and overallocation separatly.

Disable overallocation in CJK decoders: only set the minimum length.

Other changes:

 * Add min_char character to _PyUnicodeWriter. It is currenctly unused. Using _PyUnicodeWriter_Prepare(writer, 0, min_char) is different because it allocates immediatly the buffer, and calling _PyUnicodeWriter_Prepare() with size=0 is not supported (it does not widen the buffer if maxchar is bigger).
 * unicode_decode_call_errorhandler_writer() only enables overallocation if the replaced string is longer than 1 character
 * PyUnicode_DecodeRawUnicodeEscape() and _PyUnicode_DecodeUnicodeInternal() set minimum length instead of preallocating the whole buffer. It avoids the need of widen the buffer if the first written character is the biggest character. It also avoids an useless memory allocation if the decoder fails before the first write.
 * _PyUnicode_DecodeUnicodeInternal() checks for integer overflow when computing the minimum length
 * _PyUnicodeWriter_Update() is now responsible to set size to zero if readonly is set

The goal is to delay the first allocation until the first real write to be able to choose correctly the maximum character and the buffer size. If the buffer is allocated before the first write, even the first write must widen and/or enlarge the buffer.
History
Date User Action Args
2013-04-14 01:38:56vstinnersetrecipients: + vstinner, serhiy.storchaka, vladistan
2013-04-14 01:38:55vstinnersetmessageid: <1365903535.72.0.0813124801856.issue17694@psf.upfronthosting.co.za>
2013-04-14 01:38:55vstinnerlinkissue17694 messages
2013-04-14 01:38:55vstinnercreate