Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

importlib.metadata.DeprecatedList appears to be missing __slots__ #90404

Closed
ariebovenberg mannequin opened this issue Jan 3, 2022 · 7 comments
Closed

importlib.metadata.DeprecatedList appears to be missing __slots__ #90404

ariebovenberg mannequin opened this issue Jan 3, 2022 · 7 comments
Labels
3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir

Comments

@ariebovenberg
Copy link
Mannequin

ariebovenberg mannequin commented Jan 3, 2022

BPO 46246
Nosy @jaraco, @miss-islington, @ariebovenberg
PRs
  • bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList #30452
  • [3.10] bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList (GH-30452) #31269
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2022-03-14.07:35:48.184>
    created_at = <Date 2022-01-03.18:39:08.546>
    labels = ['library', '3.10', '3.11']
    title = 'importlib.metadata.DeprecatedList appears to be missing __slots__'
    updated_at = <Date 2022-03-14.07:35:48.183>
    user = 'https://github.com/ariebovenberg'

    bugs.python.org fields:

    activity = <Date 2022-03-14.07:35:48.183>
    actor = 'jaraco'
    assignee = 'none'
    closed = True
    closed_date = <Date 2022-03-14.07:35:48.184>
    closer = 'jaraco'
    components = ['Library (Lib)']
    creation = <Date 2022-01-03.18:39:08.546>
    creator = 'ariebovenberg'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 46246
    keywords = ['patch']
    message_count = 7.0
    messages = ['409604', '409644', '409652', '409680', '413031', '413032', '413034']
    nosy_count = 3.0
    nosy_names = ['jaraco', 'miss-islington', 'ariebovenberg']
    pr_nums = ['30452', '31269']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue46246'
    versions = ['Python 3.10', 'Python 3.11']

    @ariebovenberg
    Copy link
    Mannequin Author

    ariebovenberg mannequin commented Jan 3, 2022

    (as instructed in bpo-46244, I've created this ticket)

    The subclass EntryPoints defines slots, but its base DeprecatedList does not.

    This appears to be a mistake.

    If so, I'd like to create a PR to fix it.

    @ariebovenberg ariebovenberg mannequin added 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir labels Jan 3, 2022
    @jaraco
    Copy link
    Member

    jaraco commented Jan 4, 2022

    Perhaps it is a mistake. The __slots__ were added as a (possible premature) optimization in this conversation.

    It's not obvious to me what the danger is in defining __slots__ in a child class but not in a parent. Can you point me to resources or examples that help me understand the concern or missed expectation? The primary motivation behind defining slots is to help ensure immutability. Does the lack of slots violate that expectation?

    Convince me of the value of fixing this concern and please feel free to proceed with a PR.

    @ariebovenberg
    Copy link
    Mannequin Author

    ariebovenberg mannequin commented Jan 4, 2022

    @jaraco thanks for your quick response.

    In short: __slots__ allows class layout to be optimized by replacing the class __dict__ with specific descriptors. This results in a class where only specific attributes can be get/set.

    However, you only really get the savings from __slots__ if all base classes also implement it.

    An example:

        from pympler.asizeof import asizeof  # checks complete size of objects in memory
    
        class EmptyNoSlots: pass
    
        class EmptyWithSlots: __slots__ = ()
    
        class NoSlots:
            def __init__(self, a, b): self.a, self.b = a, b
    
        class WithSlots:
            __slots__ = ("a", "b")
            def __init__(self, a, b): self.a, self.b = a, b
    
        print(asizeof(EmptyNoSlots()))    # 152
        print(asizeof(EmptyWithSlots()))  # 32
        print(asizeof(NoSlots(1, 2)))     # 328
        print(asizeof(WithSlots(1, 2)))   # 112
    # Let's look at inheritance:
    
        class WithSlotsAndProperBaseClass(EmptyWithSlots):
            __slots__ = ("a", "b")
            def __init__(self, a, b): self.a, self.b = a, b
    
        class NoSlotsAtAll(EmptyNoSlots):
            def __init__(self, a, b): self.a, self.b = a, b
    
        class WithSlotsAndBadBaseClass(EmptyNoSlots):
            __slots__ = ("a", "b")
            def __init__(self, a, b): self.a, self.b = a, b
    
        print(asizeof(WithSlotsAndProperBaseClass(1, 2)))  # 112
        print(asizeof(NoSlotsAtAll(1, 2)))                 # 328
        print(asizeof(WithSlotsAndBadBaseClass(1, 2)))     # 232 -- oh no!

    In importlib:

                       list   \<- has slots (builtin)
       DeprecatedList(list)   \<- no \_\_slots__ defined
    

    EntryPoints(DeprecatedList) <- defines __slots__

    Besides the lost memory savings, because DeprecatedList has no slots, you can still do:

    EntryPoints().foo = 6  # setting a random attribute, not the intention.
    

    Further reading: https://stackoverflow.com/a/28059785

    @jaraco
    Copy link
    Member

    jaraco commented Jan 4, 2022

    Today I learned something. Thanks Arie. Yes, I agree that's a mistake. Perhaps the test suite should also have a test to capture the missed expectation (that __dict__ should not be present or setting attributes on EntryPoints instances should fail).

    @jaraco
    Copy link
    Member

    jaraco commented Feb 11, 2022

    I'm pretty sure both EntryPoints and DeprecatedList were introduced in Python 3.10, so 3.9 and 3.8 aren't relevant.

    @jaraco jaraco removed 3.8 only security fixes 3.9 only security fixes labels Feb 11, 2022
    @miss-islington
    Copy link
    Contributor

    New changeset dd76b3f by Arie Bovenberg in branch 'main':
    bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList (GH-30452)
    dd76b3f

    @miss-islington
    Copy link
    Contributor

    New changeset 1124ab6 by Miss Islington (bot) in branch '3.10':
    bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList (GH-30452)
    1124ab6

    @jaraco jaraco closed this as completed Mar 14, 2022
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants