This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Asyncio classes missing __slots__
Type: Stage: resolved
Components: asyncio Versions: Python 3.10
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: andrei.avk, asvetlov, bluenix, josh.r, serhiy.storchaka, yselivanov
Priority: normal Keywords:

Created on 2021-06-05 14:58 by bluenix, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (11)
msg395165 - (view) Author: Bluenix (bluenix) * Date: 2021-06-05 14:58
Most of asyncio's classes are missing a __slots__ attribute - for example Lock, Event, Condition, Semaphore, BoundedSemaphore all from locks.py; or Future, FlowControlMixin, Queue, BaseSubprocessTransport from various other parts of asyncio.
msg395166 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-06-05 15:15
What is the problem with this?

Setting __slots__ is needed if we want to save memory for creating a lot of instances. It can also be used for preventing adding arbitrary attributes and/or making weak references. Setting __slots__ in base class is required if you want to get a benefit of setting __slots__ in any of subclasses.
msg395170 - (view) Author: Bluenix (bluenix) * Date: 2021-06-05 15:55
> What is the problem with this?

The problem is that asyncio *is not* defining __slots__.

> Setting __slots__ in base class is required if you want to get a benefit of setting __slots__ in any of subclasses.

That is my use-case for this.
msg395171 - (view) Author: wyz23x2 (wyz23x2) * Date: 2021-06-05 16:21
OK, so:
>>> (1).__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__slots__'
>>> 4.5.__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'float' object has no attribute '__slots__'
>>> complex(5, 2).__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'complex' object has no attribute '__slots__'
>>> 'Hello'.__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute '__slots__'
>>> b'50'.__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'bytes' object has no attribute '__slots__'
>>> [2.72, 3.14].__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__slots__'
>>> 

Many many more.
So these *all* need __slots__???
That a major change into Python 5000.
msg395176 - (view) Author: Bluenix (bluenix) * Date: 2021-06-05 18:22
>>> (1).__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__dict__'
>>> 4.5.__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'float' object has no attribute '__dict__'
>>> 'Hello'.__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute '__dict__'
>>> b'50'.__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'bytes' object has no attribute '__dict__'
>>> [2.72, 3.14].__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__dict__'


> __slots__ allow us to explicitly declare data members (like properties) and deny the creation of __dict__ and __weakref__ (unless explicitly declared in __slots__ or available in a parent.)

From https://docs.python.org/3/reference/datamodel.html

They don't have __slots__, nor a __dict__ or __weakref__:

>>> (1).__weakref__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__weakref__'
>>> 4.5.__weakref__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'float' object has no attribute '__weakref__'
>>> 'Hello'.__weakref__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute '__weakref__'
>>> b'50'.__weakref__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'bytes' object has no attribute '__weakref__'
>>> [2.72, 3.14].__weakref__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__weakref__'

They're essentially C extension classes: https://docs.python.org/3/extending/newtypes_tutorial.html

I am not sure what this argument was meant to prove.


What would be the downside to adding __slots__ to asyncio's classes? Other than that someone can no longer arbitrarily add attributes?
msg395969 - (view) Author: Andrei Kulakov (andrei.avk) * (Python triager) Date: 2021-06-17 03:05
Bluenix, can you describe your use case in more detail?
msg395990 - (view) Author: Bluenix (bluenix) * Date: 2021-06-17 10:37
My exact use-case is that I am subclassing asyncio.Semaphore to change some functionality (override `release()` to do nothing and set up tasks to schedule calls to reset the counter). I am expecting *a lot* of these instances so (like Serhiy Storchaka nicely put it) I would like to reduce the memory footprint of these classes by using __slots__. The issue now becomes that asyncio.Semaphore (like most other asyncio classes) have not defined __slots__, this prohibits my subclass from taking advantage of __slots__.
msg396030 - (view) Author: Andrei Kulakov (andrei.avk) * (Python triager) Date: 2021-06-18 03:46
The size of an instance of Semaphore is 48, of an empty tuple is 40, of a small int is 28, of an instance of a normal class with a single slot in __slots__ is also 40, are you use 48 byte size will really be an issue for you?
msg396031 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2021-06-18 03:57
Andrei: The size of an instance of Semaphore is 48 bytes + 104 more bytes for the __dict__ containing its three attributes (ignoring the cost of the attributes themselves). A slotted class with three attributes only needs 56 bytes of overhead per-instance (it has no __dict__, so the 56 is the total cost). Dropping overhead of the instances by >60% can make a difference if you're really making many thousands of them.

Personally, I think Python level classes should generally default to using __slots__ unless the classes are explicitly not for subclassing; not using __slots__ means all subclasses have their hands tied by the decision of the parent class. Perhaps explicitly opting in to __weakref__ (which __slots__ removes by default) to allow weak referencing, but it's fairly rare a class *needs* to otherwise allow the creation of arbitrary attributes.
msg396033 - (view) Author: Andrei Kulakov (andrei.avk) * (Python triager) Date: 2021-06-18 04:55
My mistake, I forgot the size of the dict itself is not included in getsizeof().
msg415019 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-12 23:30
1. Guido van Rossum explicitly designed asyncio to *don't* use slots.
2. Changing it produces potential backward incompatibility issues.

Very many stdlib classes don't have slots, as already mentioned.
The default is really the reverse: no slots.

Closing.
History
Date User Action Args
2022-04-11 14:59:46adminsetgithub: 88484
2022-03-12 23:30:59asvetlovsetstatus: open -> closed
resolution: rejected
messages: + msg415019

stage: resolved
2021-06-18 04:55:40andrei.avksetmessages: + msg396033
2021-06-18 03:57:38josh.rsetnosy: + josh.r
messages: + msg396031
2021-06-18 03:46:53andrei.avksetmessages: + msg396030
2021-06-17 10:37:21bluenixsetmessages: + msg395990
2021-06-17 03:05:17andrei.avksetnosy: + andrei.avk
messages: + msg395969
2021-06-05 18:22:40bluenixsetmessages: + msg395176
2021-06-05 16:23:16wyz23x2setnosy: - wyz23x2
2021-06-05 16:21:38wyz23x2setnosy: + wyz23x2
messages: + msg395171
2021-06-05 15:55:41bluenixsetmessages: + msg395170
2021-06-05 15:15:26serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg395166
2021-06-05 14:58:52bluenixcreate