Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_Pickler_New() doesn't call PyObject_GC_Track(self) #62572

Closed
tiran opened this issue Jul 5, 2013 · 14 comments
Closed

_Pickler_New() doesn't call PyObject_GC_Track(self) #62572

tiran opened this issue Jul 5, 2013 · 14 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error

Comments

@tiran
Copy link
Member

tiran commented Jul 5, 2013

BPO 18372
Nosy @terryjreedy, @vstinner, @tiran, @benjaminp, @methane, @serhiy-storchaka, @ZackerySpytz, @miss-islington
PRs
  • bpo-18372: Add missing PyObject_GC_Track() calls in the pickle module #8505
  • [3.7] bpo-18372: Add missing PyObject_GC_Track() calls in the pickle module (GH-8505) #12926
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-04-23.12:25:18.673>
    created_at = <Date 2013-07-05.22:53:10.665>
    labels = ['extension-modules', '3.8', 'type-bug', '3.7']
    title = "_Pickler_New() doesn't call PyObject_GC_Track(self)"
    updated_at = <Date 2019-04-23.12:25:18.673>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2019-04-23.12:25:18.673>
    actor = 'methane'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-04-23.12:25:18.673>
    closer = 'methane'
    components = ['Extension Modules']
    creation = <Date 2013-07-05.22:53:10.665>
    creator = 'christian.heimes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 18372
    keywords = ['patch']
    message_count = 14.0
    messages = ['192369', '228604', '228610', '228613', '228617', '228619', '228621', '228622', '339604', '339642', '339724', '339726', '340718', '340719']
    nosy_count = 8.0
    nosy_names = ['terry.reedy', 'vstinner', 'christian.heimes', 'benjamin.peterson', 'methane', 'serhiy.storchaka', 'ZackerySpytz', 'miss-islington']
    pr_nums = ['8505', '12926']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue18372'
    versions = ['Python 3.7', 'Python 3.8']

    @tiran
    Copy link
    Member Author

    tiran commented Jul 5, 2013

    In _Pickler_New() PicklerObject is allocated with

       self = PyObject_GC_New(PicklerObject, &Pickler_Type);

    but PyObject_GC_Track(self) is never called, same for _Unpickler_New(). I think it's a bug.

    @tiran tiran added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels Jul 5, 2013
    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Oct 5, 2014

    @Christian/Victor could either of you provide a patch for this, it's way beyond my knowledge I'm afraid.

    @benjaminp
    Copy link
    Contributor

    Mark, could you please stop touching every issue on the tracker? I appreciate the effort, but giving thoughtful feedback, patches, or reviews on a just a few issues would be much more helpful.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Oct 5, 2014

    Sorry but no, when I started out on this a couple of months ago there were over 600 issues that nobody had even bothered to reply to. That number is now down to 369. I believe that around 200 issues have been closed as a result of my efforts. Do you have a problem with me showing just how easy it is to get issues resolved?

    @terryjreedy
    Copy link
    Member

    Mark, I also appreciate your efforts, as I hope should be obvious from my prior responses to many of your other posts. I also agree that somewhat fewer, higher quality posts would be even more helpful. Please do not slip into a 'confrontational' mode, as you did a few years ago. And please lets discuss this further by private email.

    @vstinner
    Copy link
    Member

    vstinner commented Oct 5, 2014

    Le 5 oct. 2014 20:04, "Mark Lawrence" <report@bugs.python.org> a écrit :

    Mark Lawrence added the comment:

    @Christian/Victor could either of you provide a patch for this, it's way
    beyond my knowledge I'm afraid.

    I agree with Benjamin. Asking directly two developers to write a patch
    don't add any value to the issue, it only puts pressure on them. If I
    didn't write a patch before, it's probably because I'm not really
    interested by the topic.

    If you would like to contribute, you can close directly issues, test and
    review patches, suggest an implementation, etc.

    It's not all black or all white. Sometimes, your "ping" messages are
    helpful reminders to finish the work or simply to close the issue.

    By experience, most old issues are not finished because the bug only impact
    a few people and it's possible to work around it, or because the feature
    request is not interesting enough.

    I didn't read this issue (i'm replying in my mail client), my remarks are
    general.

    @vstinner
    Copy link
    Member

    vstinner commented Oct 5, 2014

    I don't know the PyObject_GC_Track() function. Why is it an issue to not call it? Can you elaborate Christian? What do you suggest?

    @serhiy-storchaka
    Copy link
    Member

    From Doc/c-api/gcsupport.rst:

    Constructors for container types must conform to two rules:

    #. The memory for the object must be allocated using :c:func:`PyObject_GC_New`
    or :c:func:`PyObject_GC_NewVar`.

    #. Once all the fields which may contain references to other containers are

    initialized, it must call :c:func:`PyObject_GC_Track`.

    _pickle.Pickler and _pickle.Unpickler have the Py_TPFLAGS_HAVE_GC flag, implement tp_traverse and tp_clear, but PyObject_GC_Track is newer called.

    @ZackerySpytz ZackerySpytz mannequin added 3.7 (EOL) end of life 3.8 only security fixes labels Jul 27, 2018
    @vstinner
    Copy link
    Member

    vstinner commented Apr 8, 2019

    It has been decided to not fix the issue in Python 2.7:
    #8505 (comment)

    @serhiy-storchaka
    Copy link
    Member

    Tracking objects that do not need this will just add work to the garbage collector. Not all instances of trackable types should be tracked, for example the empty tuple and some dicts are not tracked.

    >>> gc.is_tracked(())
    False
    >>> gc.is_tracked((1, 2))
    True
    >>> gc.is_tracked({1: None})
    False
    >>> gc.is_tracked({1: []})
    True

    @vstinner
    Copy link
    Member

    vstinner commented Apr 9, 2019

    Tracking objects that do not need this will just add work to the garbage collector. Not all instances of trackable types should be tracked, for example the empty tuple and some dicts are not tracked.

    Well, in that case, we should do the opposite of PR 8505, what I proposed there:

    #8505 (comment)

    "Either GC support must be removed (remove Py_TPFLAGS_HAVE_GC, remove tp_clear and tp_traverse, etc.), or the implementation should be fixed (call PyObject_GC_Track)."

    => fully remove the GC support

    I don't see the point of implementing tp_traverse if it's not called.

    I'm not sure if tp_clear is related to the GC or not. Maybe keep it :-)

    @serhiy-storchaka
    Copy link
    Member

    GC support can not be removed, because these objects are created also as long living objects using the standard way. And in this case they are tracked (in PyType_GenericAlloc()) and tp_traverse is called. They are not tracked only when created by an internal API _Pickler_New() and _Unpickler_New().

    The current code is correct and I do not see reasons to change it.

    @methane
    Copy link
    Member

    methane commented Apr 23, 2019

    New changeset 359bd4f by Inada Naoki (Zackery Spytz) in branch 'master':
    bpo-18372: Add missing PyObject_GC_Track() calls in the pickle module (GH-8505)
    359bd4f

    @miss-islington
    Copy link
    Contributor

    New changeset c0f6f53 by Miss Islington (bot) in branch '3.7':
    bpo-18372: Add missing PyObject_GC_Track() calls in the pickle module (GH-8505)
    c0f6f53

    @methane methane closed this as completed Apr 23, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    7 participants