Author vstinner
Recipients methane, serhiy.storchaka, vstinner
Date 2020-06-10.16:39:45
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1591807186.71.0.43220674354.issue40943@roundup.psfhosted.org>
In-reply-to
Content
I started to write a long rationale to explain why "#" variants of formats must be deprecated, before I noticed that they are already deprecated! To not lost it, here is my rationale :-)


The C code base of Python switched from int to Py_ssize_t with PEP 353. For not break the backward compatibility, C extension modules have to opt-in for Py_ssize_t by defining PY_SSIZE_T_CLEAN macro in their code base. It affects a few format PyArg_ParseTuple() and Py_BuildValue():
https://docs.python.org/dev/c-api/arg.html

Currently, the documentation says "This behavior will change in a future Python version to only support Py_ssize_t and drop int support. It is best to always define PY_SSIZE_T_CLEAN."

Continue to use int by default prevents to accept content larger than 2 GB and causes issues with limited C API: bpo-27499.

I propose to raise a DeprecationWarning on C extension modules built without the PY_SSIZE_T_CLEAN macro defined.

#warning could be used to emit a warning when building a C extension without PY_SSIZE_T_CLEAN. The problem is that the compiler output is hidden by "pip install", only few developers pay attention to such warnings, and also we should not bother C extensions which don't use PyArg_ParseTuple() and Py_BuildValue().

I prefer to raise a DeprecationWarning exception at runtime since this warning can be made an error when using -Werror: it eases detection of deprecated modules without preventing to build or use a C extension using the deprecated functions.

Two releases after the DeprecationWarning is introduced, we can consider to always raise an exception. Again, I dislike the idea of always require PY_SSIZE_T_CLEAN macro when including "Python.h" header file. I suggest to only raise an exception at runtime when the few affected functions are called.

For example, PyArg_ParseTuple(args, "y*", &buffer) always Py_buffer which uses Py_ssize_t: the behavior does not depend on PY_SSIZE_T_CLEAN.

An alternative is to even deprecate the few "#" formats which are affected by PY_SSIZE_T_CLEAN.
History
Date User Action Args
2020-06-10 16:39:46vstinnersetrecipients: + vstinner, methane, serhiy.storchaka
2020-06-10 16:39:46vstinnersetmessageid: <1591807186.71.0.43220674354.issue40943@roundup.psfhosted.org>
2020-06-10 16:39:46vstinnerlinkissue40943 messages
2020-06-10 16:39:45vstinnercreate