classification
Title: Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode are not available with limited API
Type: behavior Stage: resolved
Components: C API Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2020-10-09 19:48 by serhiy.storchaka, last changed 2020-10-10 19:28 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22621 merged serhiy.storchaka, 2020-10-09 19:54
PR 22637 merged miss-islington, 2020-10-10 14:10
PR 22638 merged miss-islington, 2020-10-10 14:10
Messages (7)
msg378340 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-09 19:48
Py_FileSystemDefaultEncodeErrors was added to limited API in 3.7 and Py_UTF8Mode -- in 3.7. But in 3.8 their declarations are not available if Py_LIMITED_API is defined because they were moved to different header file included only if Py_LIMITED_API is not defined.
msg378399 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-10 13:31
I consider the global configuration variables as deprecated. I would prefer to slowly move towards PyConfig, since almost all Python functions now use PyInterpreterState.config.

The problem is that my idea to provide a stable ABI for PyConfig was rejected:
https://mail.python.org/archives/list/python-dev@python.org/thread/C7Z2NA2DTM3DLOZCFQAK5A2WFYO3PHHX/#2JAJQA5OANFPXAJ3327RRPHPQLKVP2EW

Which problem are you trying to solve your PR 22621?
msg378401 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-10 13:38
It solves the breaking of the C API. Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode were not deprecated, they were just suddenly excluded from the limited API. And seems this change was not intentional, otherwise surrounding #ifdef/#endif (which currently do not have effect) would be not preserved.
msg378404 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-10 13:52
Python 3.7 defines it in fileobject.h as:

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03060000
PyAPI_DATA(const char *) Py_FileSystemDefaultEncodeErrors;
#endif

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03070000
PyAPI_DATA(int) Py_UTF8Mode;
#endif


Python 3.8 defines them the same way, but in Include/cpython/fileobject.h:

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03060000
PyAPI_DATA(const char *) Py_FileSystemDefaultEncodeErrors;
#endif

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03070000
PyAPI_DATA(int) Py_UTF8Mode;
#endif


I am likely the one who moved these definitions. It was a mistake to move them inside Include/cpython/, since "Py_LIMITED_API+0 >= 0x03070000" became useless: cpython/fileobject.h is only included if Py_LIMITED_API is not defined.

In Include/cpython/, it seems like only 2 definitions are defined the wrong way, Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode.
msg378405 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-10 14:09
New changeset 637a09b0d6e3ad4e34e0b5e0fc82f5afeae6f74b by Serhiy Storchaka in branch 'master':
bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621)
https://github.com/python/cpython/commit/637a09b0d6e3ad4e34e0b5e0fc82f5afeae6f74b
msg378407 - (view) Author: miss-islington (miss-islington) Date: 2020-10-10 14:28
New changeset ff6870f199511f09304e9d3ee7e7d8ed3902ffd1 by Miss Skeleton (bot) in branch '3.8':
bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621)
https://github.com/python/cpython/commit/ff6870f199511f09304e9d3ee7e7d8ed3902ffd1
msg378408 - (view) Author: miss-islington (miss-islington) Date: 2020-10-10 14:32
New changeset ebc5a6b59ece48b490987bdaa2af842c29f5b2f8 by Miss Skeleton (bot) in branch '3.9':
bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621)
https://github.com/python/cpython/commit/ebc5a6b59ece48b490987bdaa2af842c29f5b2f8
History
Date User Action Args
2020-10-10 19:28:11serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-10-10 14:32:13miss-islingtonsetmessages: + msg378408
2020-10-10 14:28:53miss-islingtonsetmessages: + msg378407
2020-10-10 14:10:08miss-islingtonsetpull_requests: + pull_request21612
2020-10-10 14:10:00miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21611
2020-10-10 14:09:52serhiy.storchakasetmessages: + msg378405
2020-10-10 13:52:49vstinnersetmessages: + msg378404
2020-10-10 13:38:40serhiy.storchakasetmessages: + msg378401
2020-10-10 13:31:19vstinnersetmessages: + msg378399
2020-10-09 19:54:16serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request21601
2020-10-09 19:48:03serhiy.storchakacreate