Issue44318
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2021-06-05 14:58 by bluenix, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Messages (11) | |||
---|---|---|---|
msg395165 - (view) | Author: Bluenix (bluenix) * | Date: 2021-06-05 14:58 | |
Most of asyncio's classes are missing a __slots__ attribute - for example Lock, Event, Condition, Semaphore, BoundedSemaphore all from locks.py; or Future, FlowControlMixin, Queue, BaseSubprocessTransport from various other parts of asyncio. |
|||
msg395166 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2021-06-05 15:15 | |
What is the problem with this? Setting __slots__ is needed if we want to save memory for creating a lot of instances. It can also be used for preventing adding arbitrary attributes and/or making weak references. Setting __slots__ in base class is required if you want to get a benefit of setting __slots__ in any of subclasses. |
|||
msg395170 - (view) | Author: Bluenix (bluenix) * | Date: 2021-06-05 15:55 | |
> What is the problem with this? The problem is that asyncio *is not* defining __slots__. > Setting __slots__ in base class is required if you want to get a benefit of setting __slots__ in any of subclasses. That is my use-case for this. |
|||
msg395171 - (view) | Author: wyz23x2 (wyz23x2) * | Date: 2021-06-05 16:21 | |
OK, so: >>> (1).__slots__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'int' object has no attribute '__slots__' >>> 4.5.__slots__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'float' object has no attribute '__slots__' >>> complex(5, 2).__slots__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'complex' object has no attribute '__slots__' >>> 'Hello'.__slots__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'str' object has no attribute '__slots__' >>> b'50'.__slots__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'bytes' object has no attribute '__slots__' >>> [2.72, 3.14].__slots__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'list' object has no attribute '__slots__' >>> Many many more. So these *all* need __slots__??? That a major change into Python 5000. |
|||
msg395176 - (view) | Author: Bluenix (bluenix) * | Date: 2021-06-05 18:22 | |
>>> (1).__dict__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'int' object has no attribute '__dict__' >>> 4.5.__dict__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'float' object has no attribute '__dict__' >>> 'Hello'.__dict__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'str' object has no attribute '__dict__' >>> b'50'.__dict__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'bytes' object has no attribute '__dict__' >>> [2.72, 3.14].__dict__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'list' object has no attribute '__dict__' > __slots__ allow us to explicitly declare data members (like properties) and deny the creation of __dict__ and __weakref__ (unless explicitly declared in __slots__ or available in a parent.) From https://docs.python.org/3/reference/datamodel.html They don't have __slots__, nor a __dict__ or __weakref__: >>> (1).__weakref__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'int' object has no attribute '__weakref__' >>> 4.5.__weakref__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'float' object has no attribute '__weakref__' >>> 'Hello'.__weakref__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'str' object has no attribute '__weakref__' >>> b'50'.__weakref__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'bytes' object has no attribute '__weakref__' >>> [2.72, 3.14].__weakref__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'list' object has no attribute '__weakref__' They're essentially C extension classes: https://docs.python.org/3/extending/newtypes_tutorial.html I am not sure what this argument was meant to prove. What would be the downside to adding __slots__ to asyncio's classes? Other than that someone can no longer arbitrarily add attributes? |
|||
msg395969 - (view) | Author: Andrei Kulakov (andrei.avk) * | Date: 2021-06-17 03:05 | |
Bluenix, can you describe your use case in more detail? |
|||
msg395990 - (view) | Author: Bluenix (bluenix) * | Date: 2021-06-17 10:37 | |
My exact use-case is that I am subclassing asyncio.Semaphore to change some functionality (override `release()` to do nothing and set up tasks to schedule calls to reset the counter). I am expecting *a lot* of these instances so (like Serhiy Storchaka nicely put it) I would like to reduce the memory footprint of these classes by using __slots__. The issue now becomes that asyncio.Semaphore (like most other asyncio classes) have not defined __slots__, this prohibits my subclass from taking advantage of __slots__. |
|||
msg396030 - (view) | Author: Andrei Kulakov (andrei.avk) * | Date: 2021-06-18 03:46 | |
The size of an instance of Semaphore is 48, of an empty tuple is 40, of a small int is 28, of an instance of a normal class with a single slot in __slots__ is also 40, are you use 48 byte size will really be an issue for you? |
|||
msg396031 - (view) | Author: Josh Rosenberg (josh.r) * | Date: 2021-06-18 03:57 | |
Andrei: The size of an instance of Semaphore is 48 bytes + 104 more bytes for the __dict__ containing its three attributes (ignoring the cost of the attributes themselves). A slotted class with three attributes only needs 56 bytes of overhead per-instance (it has no __dict__, so the 56 is the total cost). Dropping overhead of the instances by >60% can make a difference if you're really making many thousands of them. Personally, I think Python level classes should generally default to using __slots__ unless the classes are explicitly not for subclassing; not using __slots__ means all subclasses have their hands tied by the decision of the parent class. Perhaps explicitly opting in to __weakref__ (which __slots__ removes by default) to allow weak referencing, but it's fairly rare a class *needs* to otherwise allow the creation of arbitrary attributes. |
|||
msg396033 - (view) | Author: Andrei Kulakov (andrei.avk) * | Date: 2021-06-18 04:55 | |
My mistake, I forgot the size of the dict itself is not included in getsizeof(). |
|||
msg415019 - (view) | Author: Andrew Svetlov (asvetlov) * | Date: 2022-03-12 23:30 | |
1. Guido van Rossum explicitly designed asyncio to *don't* use slots. 2. Changing it produces potential backward incompatibility issues. Very many stdlib classes don't have slots, as already mentioned. The default is really the reverse: no slots. Closing. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:46 | admin | set | github: 88484 |
2022-03-12 23:30:59 | asvetlov | set | status: open -> closed resolution: rejected messages: + msg415019 stage: resolved |
2021-06-18 04:55:40 | andrei.avk | set | messages: + msg396033 |
2021-06-18 03:57:38 | josh.r | set | nosy:
+ josh.r messages: + msg396031 |
2021-06-18 03:46:53 | andrei.avk | set | messages: + msg396030 |
2021-06-17 10:37:21 | bluenix | set | messages: + msg395990 |
2021-06-17 03:05:17 | andrei.avk | set | nosy:
+ andrei.avk messages: + msg395969 |
2021-06-05 18:22:40 | bluenix | set | messages: + msg395176 |
2021-06-05 16:23:16 | wyz23x2 | set | nosy:
- wyz23x2 |
2021-06-05 16:21:38 | wyz23x2 | set | nosy:
+ wyz23x2 messages: + msg395171 |
2021-06-05 15:55:41 | bluenix | set | messages: + msg395170 |
2021-06-05 15:15:26 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg395166 |
2021-06-05 14:58:52 | bluenix | create |