classification
Title: Rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString(): don't cache the result
Type: enhancement Stage:
Components: Versions: Python 3.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: loewis, pitrou, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2014-09-01 21:32 by vstinner, last changed 2018-07-23 20:12 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
unicode_aswidechar.patch vstinner, 2014-09-02 07:52
Messages (5)
msg226244 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-01 21:32
I would like to deprecate PyUnicode_AsUnicode(), see the issue #22271 for the rationale (hint: memory footprint). The first step is to rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString() to not call PyUnicode_AsUnicode() anymore.

Attached patch implements this.

The code is based on PyUnicode_AsUnicode(), but it's more tricky because PyUnicode_AsWideChar() can truncate the string, and PyUnicode_AsUnicode() does no copy characters if kind == sizeof(wchar_t), PyASCIIObject.wstr "just" points to data.

I hate PyUnicode_AsWideChar(), but we must keep it for backward compatibility :-)

It would be possible to write an optimized PyUnicode_AsWideCharString() which computes the length, allocate memory and write wide characters, but I don't want to have 3 functions converting a Python string to a wide character string. There are already PyUnicode_AsUnicodeAndSize() and unicode_aswidechar() (+ unicode_aswidechar_len()).
msg226260 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-02 02:37
> Attached patch implements this.

There is no patch.
msg226264 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-02 07:52
> There is no patch.

You're right. Here it is.
msg226515 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-06 20:17
Hmm... sorry for the delay, there's no review link. Perhaps the patch is not against the latest default?
msg322253 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-23 20:12
Oh, I have reimplemented this in issue30863.
History
Date User Action Args
2018-07-23 20:12:05serhiy.storchakasetmessages: + msg322253
2015-10-02 21:06:09vstinnersetstatus: open -> closed
resolution: wont fix
2014-10-09 09:08:27vstinnersettitle: Rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString() -> Rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString(): don't cache the result
2014-09-06 20:17:56pitrousetmessages: + msg226515
2014-09-04 10:47:28vstinnersetnosy: + serhiy.storchaka
2014-09-02 07:52:30vstinnersetfiles: + unicode_aswidechar.patch
keywords: + patch
messages: + msg226264
2014-09-02 02:37:57pitrousetnosy: + pitrou
messages: + msg226260
2014-09-01 21:32:14vstinnercreate