classification
Title: Deprecate PyUnicode_AsUnicode(): emit a DeprecationWarning
Type: Stage: resolved
Components: Versions: Python 3.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: loewis, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2014-08-25 01:41 by vstinner, last changed 2017-06-28 01:28 by vstinner. This issue is now closed.

Messages (5)
msg225860 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-08-25 01:41
With the PEP 393 implemented in Python 3.3, PyUnicode_AsUnicode() and all similar functions have to convert the compact string to a heavy wchar_t* string (UCS-4 on Linux: 4 bytes per character, UTF-16 on Windows: 2 or 4 bytes per character) which is stored in the string. The heavy "Py_UNICODE*" storage is kept until the string is destroyed, which may only occur at Python exit.

To reduce the memory footprint, it would be nice to promote the usage of the PEP 393 and start to emit a DeprecationWarning warning. The Py_UNICODE type and all related functions are already deprecated in the documentation.

The deprecate PyUnicode_AsUnicode(), we should stop using it in Python itself. For example, PyUnicode_AsWideCharString() can be used to encode filenames on Windows, it doesn't store the encoded string in the Python object. See for example path_converter() in posixmodule.c.
msg226245 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-01 21:42
> The deprecate PyUnicode_AsUnicode(), we should stop using it in Python itself.

Here is the first step: issue #22323, "rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString()" (to not call PyUnicode_AsUnicode() anymore).
msg226296 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-03 06:47
It would be better to emit a warning only at compile time. End user of compiled extension can't do anything with a warning emitted at run time, only its author can avoid it.
msg226297 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-03 07:03
> It would be better to emit a warning only at compile time. End user of compiled extension can't do anything with a warning emitted at run time, only its author can avoid it.

DeprecatedWarning warnings are silent by default. So it will not annoy users.
msg297115 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-06-28 01:28
Sorry I lost track of this issue, I just close it.

Note: Serhiy recently worked on wchar_t functions of PyUnicodeObject to detect embedded null characters, see for example bpo-30708.
History
Date User Action Args
2017-06-28 01:28:55vstinnersetstatus: open -> closed
resolution: out of date
messages: + msg297115

stage: resolved
2014-09-03 07:03:36vstinnersetmessages: + msg226297
2014-09-03 06:47:48serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg226296
2014-09-01 21:42:44vstinnersetmessages: + msg226245
2014-08-25 01:41:16vstinnercreate