Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

C implementation of ZoneInfo cannot be subclassed #85197

Closed
pganssle opened this issue Jun 18, 2020 · 11 comments
Closed

C implementation of ZoneInfo cannot be subclassed #85197

pganssle opened this issue Jun 18, 2020 · 11 comments
Assignees
@pganssle
Labels
3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir

Comments

@pganssle
Copy link
Member

BPO 41025
Nosy @vstinner, @ambv, @pganssle, @miss-islington
PRs
  • bpo-41025: Fix subclassing for zoneinfo.ZoneInfo #20965
  • [3.9] bpo-41025: Fix subclassing for zoneinfo.ZoneInfo (GH-20965) #21876
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/pganssle'
closed_at = <Date 2020-09-14.17:50:17.320>
created_at = <Date 2020-06-18.14:26:51.163>
labels = ['library', '3.9', '3.10']
title = 'C implementation of ZoneInfo cannot be subclassed'
updated_at = <Date 2020-09-14.17:50:17.320>
user = 'https://github.com/pganssle'

    bugs.python.org fields:

    activity = <Date 2020-09-14.17:50:17.320>
actor = 'lukasz.langa'
assignee = 'p-ganssle'
closed = True
closed_date = <Date 2020-09-14.17:50:17.320>
closer = 'lukasz.langa'
components = ['Library (Lib)']
creation = <Date 2020-06-18.14:26:51.163>
creator = 'p-ganssle'
dependencies = []
files = []
hgrepos = []
issue_num = 41025
keywords = ['patch']
message_count = 11.0
messages = ['371817', '375365', '375412', '375418', '375419', '375530', '375559', '375657', '376712', '376730', '376886']
nosy_count = 4.0
nosy_names = ['vstinner', 'lukasz.langa', 'p-ganssle', 'miss-islington']
pr_nums = ['20965', '21876']
priority = 'critical'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41025'
versions = ['Python 3.9', 'Python 3.10']
    @pganssle
    Copy link
    Member Author

    In the C implementation of zoneinfo.ZoneInfo, __init_subclass__ is not declared as a classmethod, which prevents it from being subclassed. This was not noticed because the tests for ZoneInfo subclasses in C are actually testing zoneinfo.ZoneInfo, not a subclass, due to a mistake in the inheritance tree:

    cpython/Lib/test/test_zoneinfo/test_zoneinfo.py

    Lines 465 to 487 in 8f192d1

    class ZoneInfoTestSubclass(ZoneInfoTest):
    @classmethod
    def setUpClass(cls):
    super().setUpClass()
    class ZISubclass(cls.klass):
    pass
    cls.class_name = "ZISubclass"
    cls.parent_klass = cls.klass
    cls.klass = ZISubclass
    def test_subclass_own_cache(self):
    base_obj = self.parent_klass("Europe/London")
    sub_obj = self.klass("Europe/London")
    self.assertIsNot(base_obj, sub_obj)
    self.assertIsInstance(base_obj, self.parent_klass)
    self.assertIsInstance(sub_obj, self.klass)
    class CZoneInfoTestSubclass(ZoneInfoTest):
    module = c_zoneinfo

    Originally reported on the backport by Sébastien Eustace: pganssle/zoneinfo#82

    The fix in the backport is here: pganssle/zoneinfo#83

    @pganssle pganssle added 3.9 only security fixes 3.10 only security fixes labels Jun 18, 2020
    @pganssle pganssle self-assigned this Jun 18, 2020
    @pganssle pganssle added stdlib Python modules in the Lib dir 3.9 only security fixes 3.10 only security fixes labels Jun 18, 2020
    @pganssle pganssle self-assigned this Jun 18, 2020
    @pganssle pganssle added the stdlib Python modules in the Lib dir label Jun 18, 2020
    @pganssle
    Copy link
    Member Author

    New changeset 87d8287 by Paul Ganssle in branch 'master':
    bpo-41025: Fix subclassing for zoneinfo.ZoneInfo (GH-20965)
    87d8287

    @pganssle
    Copy link
    Member Author

    New changeset 33d3c64 by Miss Islington (bot) in branch '3.9':
    bpo-41025: Fix subclassing for zoneinfo.ZoneInfo (GH-20965) (GH-21876)
    33d3c64

    @pganssle
    Copy link
    Member Author

    Łukasz: Would it be possible to backport / cherry-pick the changes from PR #66075 (#21876) into the 3.9.0 branch? I think this is a fairly severe issue considering that pendulum is planning to use a zoneinfo subclass.

    It was merged as commit 33d3c64.

    Thanks!

    @pganssle
    Copy link
    Member Author

    Marking as release blocker to put it on the checklist. Feel free to demote it if you decide it should be deferred to 3.9.1.

    @vstinner
    Copy link
    Member

    New changeset 87d8287 by Paul Ganssle in branch 'master':
    bpo-41025: Fix subclassing for zoneinfo.ZoneInfo (GH-20965)

    This change introduced a reference leak. 3.9 and master branch are affected.

    $ make && ./python -m test -R 3:3 test_zoneinfo 
(...)
test_zoneinfo leaked [84, 84, 84] references, sum=252
test_zoneinfo leaked [41, 41, 41] memory blocks, sum=123
(...)

    ZoneInfoSubclassTest and CZoneInfoSubclassTest test cases leak.

    Example of test method which leaks:

    test.test_zoneinfo.test_zoneinfo.CZoneInfoSubclassTest.test_folds_and_gaps

    @pganssle
    Copy link
    Member Author

    There are two refleaks here. One is a reference leaking to the weak cache in __init_subclass__ (one leak every time a subclass is created), and the other is that when subclass.clear_cache() is called, it sets ZONEINFO_STRONG_CACHE = NULL, thus causing a reference leak to the parent class's strong cache.

    I'll send a PR to fix it shortly.

    @vstinner
    Copy link
    Member

    test_zoneinfo leaked [84, 84, 84] references, sum=252

    Reported as bpo-41568 and fixed there.

    @vstinner
    Copy link
    Member

    Paul: What is the status of this issue? Is it already fixed? It is the release blocker priority.

    @pganssle
    Copy link
    Member Author

    This is fixed in the 3.9 and master branches, it needs to be cherry-picked into the 3.9.0 release (at Łukasz's discretion, of course).

    Łukasz: If you cherry-pick #65164/GH-21876 into the 3.9.0 release, please also cherry-pick #66106/GH-21912, since that fixes the refleak. (PRs are listed as "master"/"backport" since I don't know your workflow).

    @ambv
    Copy link
    Contributor

    ambv commented Sep 14, 2020

    I will be cutting RC2 from the head of 3.9.

    @ambv ambv closed this as completed Sep 14, 2020
    @ambv ambv closed this as completed Sep 14, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @pganssle pganssle

    Labels
    3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @ambv @vstinner @pganssle