Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 573: Module State Access from C Extension Methods #82968

Closed
Dormouse759 mannequin opened this issue Nov 13, 2019 · 18 comments
Closed

PEP 573: Module State Access from C Extension Methods #82968

Dormouse759 mannequin opened this issue Nov 13, 2019 · 18 comments
Labels
3.9 only security fixes extension-modules C modules in the Modules dir type-feature A feature request or enhancement

Comments

@Dormouse759
Copy link
Mannequin

Dormouse759 mannequin commented Nov 13, 2019

BPO 38787
Nosy @terryjreedy, @ncoghlan, @scoder, @vstinner, @encukou, @Dormouse759, @miss-islington, @shihai1991
PRs
  • [PEP 573] bpo-38787: Module State Access from C Extension Methods #17145
  • bpo-38787: C API for module state access from extension methods (PEP 573) #19936
  • bpo-38787: update docs of structures.rst and code style of _testmultiphase.c #19980
  • bpo-38787: Add PyCFunction_CheckExact() macro for exact type checks #20024
  • bpo-38787: Fix Argument Clinic defining_class_converter #20074
  • bpo-38787: Clarify docs for PyType_GetModule and warn against common mistake #20215
  • Remove a redundant NULL in descrobject.c #20344
  • [3.9] bpo-38787: Clarify docs for PyType_GetModule and warn against common mistake (GH-20215) #21984
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-10-20.08:56:18.304>
    created_at = <Date 2019-11-13.14:30:40.496>
    labels = ['extension-modules', 'type-feature', '3.9']
    title = 'PEP 573: Module State Access from C Extension Methods'
    updated_at = <Date 2020-11-25.17:08:25.229>
    user = 'https://github.com/Dormouse759'

    bugs.python.org fields:

    activity = <Date 2020-11-25.17:08:25.229>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-10-20.08:56:18.304>
    closer = 'petr.viktorin'
    components = ['Extension Modules']
    creation = <Date 2019-11-13.14:30:40.496>
    creator = 'Dormouse759'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38787
    keywords = ['patch']
    message_count = 18.0
    messages = ['356533', '367552', '367569', '367636', '368180', '368337', '368356', '368577', '368580', '368733', '368735', '368768', '368789', '368804', '375991', '376411', '379107', '381853']
    nosy_count = 8.0
    nosy_names = ['terry.reedy', 'ncoghlan', 'scoder', 'vstinner', 'petr.viktorin', 'Dormouse759', 'miss-islington', 'shihai1991']
    pr_nums = ['17145', '19936', '19980', '20024', '20074', '20215', '20344', '21984']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue38787'
    versions = ['Python 3.9']

    @Dormouse759
    Copy link
    Mannequin Author

    Dormouse759 mannequin commented Nov 13, 2019

    Currently, there is not way to access per-module state from the methods of types defined in extension modules.

    This PEP provides a fix to this issue.

    PEP-573 - https://www.python.org/dev/peps/pep-0573/

    Reference implementation - https://github.com/Dormouse759/cpython/tree/pep-c-rebase_newer

    @Dormouse759 Dormouse759 mannequin added 3.9 only security fixes extension-modules C modules in the Modules dir type-feature A feature request or enhancement labels Nov 13, 2019
    @scoder
    Copy link
    Contributor

    scoder commented Apr 28, 2020

    What can we do to move this forward? I see that the original PR was closed in January. Is there or will there be a new one? Can I help with anything?

    @terryjreedy
    Copy link
    Member

    The PEP is listed as accepted, with you as the BDFL-delegate, but it lacks a link to the public acceptance message (usually on pydev list).

    The PR was not rejected by a core dev but was closed without explanation by the author on the same day the last commits were added. Perhaps OP just gave up getting tests to pass. But they disappear on closes.

    However, the contribution had been made under the CLA and you or Petr or any other core dev should be able to download it to a local branch, try to whip it into shape, and make a new PR before beta 1, in about a month. Or start from scratch.

    I cannot test downloading at the moment. Apparently, OP's fork branch has been altered, so that merely reopening is not an option. The OP used force-pushes rather than update merges, and this often creates problems.

    @encukou
    Copy link
    Member

    encukou commented Apr 29, 2020

    I'm working on the implementation as my time allows, here: encukou#3 .

    Would it help to create a proper PR for CPython now? That would mean I could no longer do rebases/force-pushes, so the commits would be less readable. But happy to do it if it would help you.

    @vstinner
    Copy link
    Member

    vstinner commented May 5, 2020

    See also bpo-40137: TODO list when PEP-573 "Module State Access from C Extension Methods" will be implemented.

    @vstinner
    Copy link
    Member

    vstinner commented May 7, 2020

    New changeset e1becf4 by Petr Viktorin in branch 'master':
    bpo-38787: C API for module state access from extension methods (PEP-573) (GH-19936)
    e1becf4

    @vstinner
    Copy link
    Member

    vstinner commented May 7, 2020

    New changeset c068b53 by Hai Shi in branch 'master':
    bpo-38787: Update structures.rst docs (PEP-573) (GH-19980)
    c068b53

    @vstinner
    Copy link
    Member

    See also bpo-40574: segfault causing regression from PEP-573 implementation (PyQt5).

    @vstinner
    Copy link
    Member

    The posix/nt and _abc modules now use the PEP-573 APIs:

    @miss-islington
    Copy link
    Contributor

    New changeset 4c9ea09 by scoder in branch 'master':
    bpo-38787: Add PyCFunction_CheckExact() macro for exact type checks (GH-20024)
    4c9ea09

    @shihai1991
    Copy link
    Member

    In PR19982, petr have mentioned that PyType_GetModule can not worked if the type is the subtype.

    I try to use PEP-573 in functools, I get the error info from testcases:
    ======================================================================
    ERROR: test_arg_combinations (test.test_functools.TestPartialCSubclass)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/temp/shihai/cpython/Lib/test/test_functools.py", line 109, in test_arg_combinations
        p = self.partial(capture)
    TypeError: PyType_GetModule: Type 'CPartialSubclass' has no associated module

    ======================================================================
    ERROR: test_attributes (test.test_functools.TestPartialCSubclass)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/temp/shihai/cpython/Lib/test/test_functools.py", line 71, in test_attributes
        p = self.partial(capture, 1, 2, a=10, b=20)
    TypeError: PyType_GetModule: Type 'CPartialSubclass' has no associated module

    @encukou
    Copy link
    Member

    encukou commented May 13, 2020

    I guess I'll need to clarify the documentation here.
    Call PyType_GetModule on the defining class, not on type(self). See the PEP for details.
    The associated module is not inherited; each class in the MRO can be defined in a different module.

    @shihai1991
    Copy link
    Member

    I guess I'll need to clarify the documentation here.
    Call PyType_GetModule on the defining class, not on type(self).

    +1, I didn't notice this detail until I got the error.

    The associated module is not inherited; each class in the MRO can be > defined in a different module.

    Thanks, pep have enough info, I need spend sometime to make sure I understand it clearly, Lol~.

    @vstinner
    Copy link
    Member

    New changeset 97e1568 by Victor Stinner in branch 'master':
    bpo-38787: Fix Argument Clinic defining_class_converter (GH-20074)
    97e1568

    @encukou
    Copy link
    Member

    encukou commented Aug 27, 2020

    New changeset d9a966a by Petr Viktorin in branch 'master':
    bpo-38787: Clarify docs for PyType_GetModule and warn against common mistake (GH-20215)
    d9a966a

    @miss-islington
    Copy link
    Contributor

    New changeset fbb9ee0 by Miss Islington (bot) in branch '3.9':
    [3.9] bpo-38787: Clarify docs for PyType_GetModule and warn against common mistake (GH-20215) (GH-21984)
    fbb9ee0

    @encukou
    Copy link
    Member

    encukou commented Oct 20, 2020

    Thanks Victor, Hai Shi, Pablo, Stefan, Dong-hee Na and Mariatta for reviews!

    I did not put all the details in the documentation because there are still feature gaps. So, a full write-up is now in the informational PEP-630.

    @encukou encukou closed this as completed Oct 20, 2020
    @vstinner
    Copy link
    Member

    FYI Cython doesn't support this PEP yet: cython/cython#3917

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes extension-modules C modules in the Modules dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants