Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark new limited C API #73244

Closed
serhiy-storchaka opened this issue Dec 24, 2016 · 13 comments
Closed

Mark new limited C API #73244

serhiy-storchaka opened this issue Dec 24, 2016 · 13 comments
Assignees
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@serhiy-storchaka
Copy link
Member

BPO 29058
Nosy @vstinner, @ericsnowcurrently, @serhiy-storchaka, @zooba
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • limited-api.patch
  • limited-api-2.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2017-01-22.19:16:06.188>
    created_at = <Date 2016-12-24.00:11:50.826>
    labels = ['interpreter-core', '3.7']
    title = 'Mark new limited C API'
    updated_at = <Date 2017-03-31.16:36:37.096>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:37.096>
    actor = 'dstufft'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2017-01-22.19:16:06.188>
    closer = 'serhiy.storchaka'
    components = ['Interpreter Core']
    creation = <Date 2016-12-24.00:11:50.826>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = ['46016', '46044']
    hgrepos = []
    issue_num = 29058
    keywords = ['patch']
    message_count = 13.0
    messages = ['283910', '284008', '284009', '284037', '284092', '284095', '284103', '284110', '284129', '284131', '284132', '286013', '286024']
    nosy_count = 5.0
    nosy_names = ['vstinner', 'python-dev', 'eric.snow', 'serhiy.storchaka', 'steve.dower']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue29058'
    versions = ['Python 3.6', 'Python 3.7']

    @serhiy-storchaka
    Copy link
    Member Author

    Functions added to a limited API after 3.2 should be available only when Py_LIMITED_API is not defined or is set to corresponding hexadecimal Python version (e.g. 0x03050000).

    Proposed patch makes following names available only for corresponding versions of a limited API.

    Removed declaration: PyErr_SetExcWithArgsKwargs().

    Excluded from stable ABI: _PyBytes_DecodeEscape(), PyInit_imp().

    3.3: Py_hexdigits, PyImport_ExecCodeModuleObject(), PyImport_AddModuleObject(), PyImport_ImportFrozenModuleObject(), PyMemoryView_FromMemory(), PyModule_NewObject(), PyModule_GetNameObject(), PyObject_GenericSetDict(), PyErr_GetExcInfo(), PyErr_SetExcInfo(), PyErr_SetImportError(), PyParser_SimpleParseStringFlagsFilename(), PyThread_GetInfo(), PyUnicode_Substring(), PyUnicode_AsUCS4(), PyUnicode_AsUCS4Copy(), PyUnicode_GetLength(), PyUnicode_ReadChar(), PyUnicode_WriteChar(), PyUnicode_DecodeCodePageStateful(), PyUnicode_EncodeCodePage(), PyUnicode_DecodeLocaleAndSize(), PyUnicode_DecodeLocale(), PyUnicode_EncodeLocale(), PyUnicode_FindChar(), and a number of OSError subclasses.

    3.4: PyErr_SetFromErrnoWithFilenameObjects(), PyErr_SetExcFromWindowsErrWithFilenameObjects().

    3.5: PyNumber_MatrixMultiply(), PyNumber_InPlaceMatrixMultiply(), PyCodec_NameReplaceErrors(), Py_DecodeLocale(), Py_EncodeLocale(), PyImport_ImportModuleLevelObject(), PyObject_Calloc(), PyExc_StopAsyncIteration, PyExc_RecursionError, PyMem_Calloc(),
    a number of PyODict_* macros.

    3.6: Py_FileSystemDefaultEncodeErrors, PyOS_FSPath(), PyExc_ModuleNotFoundError, PyErr_SetImportErrorSubclass(), PyErr_ResourceWarning().

    However it may be better that some was added to stable ABI by mistake. Py_hexdigits looks as a stuff for internal use, PyThread_GetInfo() and PyODict_* macros are not documented.

    @serhiy-storchaka serhiy-storchaka added 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Dec 24, 2016
    @zooba
    Copy link
    Member

    zooba commented Dec 25, 2016

    Those changes all look good to me, though it's a shame we can't leave some of those functions out altogether.

    @serhiy-storchaka
    Copy link
    Member Author

    Actually PyODict_Check, PyODict_CheckExact, and PyODict_SIZE should be excluded from limited API. These macros are expanded to the code that don't work with limited API (needed PyODict_Type and PyDictObject). PyODict_HasKey is expanded to syntactically invalid code.

    @serhiy-storchaka
    Copy link
    Member Author

    Updated patch removes PyODict_HasKey() at all, removes PyODict_Check(), PyODict_CheckExact(), PyODict_SIZE() and Py_hexdigits from stable API.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Dec 27, 2016

    New changeset f26c16aba11e by Serhiy Storchaka in branch '3.6':
    Issue bpo-29058: All stable API extensions added after Python 3.2 are now
    https://hg.python.org/cpython/rev/f26c16aba11e

    New changeset 77f5f31bf699 by Serhiy Storchaka in branch 'default':
    Issue bpo-29058: All stable API extensions added after Python 3.2 are now
    https://hg.python.org/cpython/rev/77f5f31bf699

    @serhiy-storchaka
    Copy link
    Member Author

    Changes are not applied to 3.5. It is harder to do since many private functions in 3.5 are still available in limited API. And there is a risk to break third-party code that defines Py_LIMITED_API, but uses API with higher version than required.

    @zooba
    Copy link
    Member

    zooba commented Dec 27, 2016

    Can we add a #warn to the headers then? So people know that just defining Py_LIMITED_API actually implies =0x03050000 (or whatever value is appropriate)?

    @serhiy-storchaka
    Copy link
    Member Author

    Just defining Py_LIMITED_API actually implies =0x03020000.

    I have no idea where a #warn can be added.

    @zooba
    Copy link
    Member

    zooba commented Dec 27, 2016

    Just defining Py_LIMITED_API actually implies =0x03020000.

    That's the intent, but if it were actually the case then this issue wouldn't exist :)

    On 3.5.3, building with Py_LIMITED_API=1 will include APIs that should only be included when Py_LIMITED_API=0x03050000. You considered fixing that too likely to break existing users (and I agree), but that doesn't mean we shouldn't make it clear that it's not doing exactly the right thing.

    I have no idea where a #warn can be added.

    pyport.h or pymacro.h are probably the best places. If you null-merge into 3.6 then we shouldn't have to worry about the warning showing up in later versions.

    @serhiy-storchaka
    Copy link
    Member Author

    Sorry, but perhaps I don't fully understand you.

    It is legitimately to just define Py_LIMITED_API without requiring specific version:

        #define Py_LIMITED_API

    In that case you can use the stable API of the version 3.2, but can't use PyType_FromSpecWithBases() and PyModule_AddFunctions(), because they are correctly attributed as API of versions 3.3 and 3.5. You can also mistakenly use PyImport_ImportModuleLevelObject() added in 3.5, this is a matter of this issue. But you shouldn't.

    The problem is that the warning should be emitted only for users that use incorrect API. But it shouldn't be emitted for users that use just 3.2 API (perhaps the code was written at the time of 3.2 and was not changed since).

    @zooba
    Copy link
    Member

    zooba commented Dec 27, 2016

    You've described it correctly.

    The problem is that the warning should be emitted only for users that use incorrect API. But it shouldn't be emitted for users that use just 3.2 API

    This is why I suggested #warn and not #error. It's okay to ignore warnings if you know what you're doing, but if there's no warning then people who don't know what they're doing will get it wrong. We know that some people are subtly broken here, and ought to tell them.

    Further, if we make the warning only appear for "defined(Py_LIMITED_API) && Py_LIMITED_API+0<0x03000000" then the warning can be suppressed by setting the exact version you intend to use (even though this doesn't prevent you from using the incorrect functions).

    @serhiy-storchaka
    Copy link
    Member Author

    Do you suggest to emit a warning when the user just defines Py_LIMITED_API without specifying concrete version?

        #define Py_LIMITED_API

    I think this would break too much code. And contradicts to the documentation.

    If you mean something different, could you provide a patch?

    @zooba
    Copy link
    Member

    zooba commented Jan 22, 2017

    I don't care enough to argue about it with you. Let's just fix the API as soon as we can and apologize to people who hit inconsistencies in earlier versions.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants