Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode are not available with limited API #86152

Closed
serhiy-storchaka opened this issue Oct 9, 2020 · 7 comments
Closed

Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode are not available with limited API #86152

serhiy-storchaka opened this issue Oct 9, 2020 · 7 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes topic-C-API type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 41986
Nosy @vstinner, @serhiy-storchaka, @miss-islington
PRs
  • bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API #22621
  • [3.9] bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621) #22637
  • [3.8] bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621) #22638
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-10-10.19:28:11.336>
created_at = <Date 2020-10-09.19:48:03.658>
labels = ['expert-C-API', 'type-bug', '3.8', '3.9', '3.10']
title = 'Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode are not available with limited API'
updated_at = <Date 2020-10-10.19:28:11.335>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2020-10-10.19:28:11.335>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-10.19:28:11.336>
closer = 'serhiy.storchaka'
components = ['C API']
creation = <Date 2020-10-09.19:48:03.658>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 41986
keywords = ['patch']
message_count = 7.0
messages = ['378340', '378399', '378401', '378404', '378405', '378407', '378408']
nosy_count = 3.0
nosy_names = ['vstinner', 'serhiy.storchaka', 'miss-islington']
pr_nums = ['22621', '22637', '22638']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41986'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
    @serhiy-storchaka
    Copy link
    Member Author

    Py_FileSystemDefaultEncodeErrors was added to limited API in 3.7 and Py_UTF8Mode -- in 3.7. But in 3.8 their declarations are not available if Py_LIMITED_API is defined because they were moved to different header file included only if Py_LIMITED_API is not defined.

    @serhiy-storchaka serhiy-storchaka added 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes topic-C-API type-bug An unexpected behavior, bug, or error labels Oct 9, 2020
    @vstinner
    Copy link
    Member

    I consider the global configuration variables as deprecated. I would prefer to slowly move towards PyConfig, since almost all Python functions now use PyInterpreterState.config.

    The problem is that my idea to provide a stable ABI for PyConfig was rejected:
    https://mail.python.org/archives/list/python-dev@python.org/thread/C7Z2NA2DTM3DLOZCFQAK5A2WFYO3PHHX/#2JAJQA5OANFPXAJ3327RRPHPQLKVP2EW

    Which problem are you trying to solve your PR 22621?

    @serhiy-storchaka
    Copy link
    Member Author

    It solves the breaking of the C API. Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode were not deprecated, they were just suddenly excluded from the limited API. And seems this change was not intentional, otherwise surrounding #ifdef/#endif (which currently do not have effect) would be not preserved.

    @vstinner
    Copy link
    Member

    Python 3.7 defines it in fileobject.h as:

    #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03060000
PyAPI_DATA(const char *) Py_FileSystemDefaultEncodeErrors;
#endif

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03070000
PyAPI_DATA(int) Py_UTF8Mode;
#endif

    Python 3.8 defines them the same way, but in Include/cpython/fileobject.h:

    #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03060000
PyAPI_DATA(const char *) Py_FileSystemDefaultEncodeErrors;
#endif

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03070000
PyAPI_DATA(int) Py_UTF8Mode;
#endif

    I am likely the one who moved these definitions. It was a mistake to move them inside Include/cpython/, since "Py_LIMITED_API+0 >= 0x03070000" became useless: cpython/fileobject.h is only included if Py_LIMITED_API is not defined.

    In Include/cpython/, it seems like only 2 definitions are defined the wrong way, Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode.

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset 637a09b by Serhiy Storchaka in branch 'master':
    bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621)
    637a09b

    @miss-islington
    Copy link
    Contributor

    New changeset ff6870f by Miss Skeleton (bot) in branch '3.8':
    bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621)
    ff6870f

    @miss-islington
    Copy link
    Contributor

    New changeset ebc5a6b by Miss Skeleton (bot) in branch '3.9':
    bpo-41986: Add Py_FileSystemDefaultEncodeErrors and Py_UTF8Mode back to limited API (GH-22621)
    ebc5a6b

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes topic-C-API type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @serhiy-storchaka @miss-islington