New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
asyncio: break some cycles #67398
Comments
Hi, I would like to submit 3 trivial modifications which break a cycle each. It is not much, but those three cycles caused a lot of objects to be garbage collected. They can now be freed using the reference counting mechanism, and therefore, reduce the latency that may be involved by the work of the garbage collector in a long living process. In asyncio/base_subprocess.py: I break the cycle in the protocol at the end of connection_lost(). In asyncio/futures.py: In the (really trivial) patch, the lambda uses the argument "future" instead of the "fut" variable defined in a closure. The closure is not needed anymore. This single cycle is very common, because caused when one uses getaddrinfo(). In asyncio/selectors.py: The reference to the map in the selector is cleared once the selector is closed. |
All three changes look good to me. The selectors.py fix should be applied to CPython first; the others to Tulip first. |
New changeset 376c5398f28d by Victor Stinner in branch '3.4': |
Hi Martin, thanks for the patch. It looks good to me. I applied it to Tulip, Python 3.4 and Python 3.5. |
New changeset 7438f2e30908 by Victor Stinner in branch '3.4': New changeset 27cbc877447b by Victor Stinner in branch 'default': |
Ooops, test_selectors fails because get_key() raises "TypeError: 'NoneType' object is not subscriptable" when the selector is closed. A different fix should be written. I'm now using tox to run the Tulip test suite, I'm surprised that I didn't notice the failure. It looks like test_selectors is not executed. runtests.py searchs for test classes with a name ending with "Tests". I will fix that. |
I updated the selector patch so BaseSelector.get_key() raises KeyError if the mapping is None. All the (non skipped) tests in test_selectors.py passed. Anyway, if there is an other problem with freeing the mapping object (I don't know, maybe "reopening" a loop may be considered?) this patch can probably be dropped. Since this cycle is broken when the loop is closed, the objects will likely be collected once the program terminates. |
I opened the issue bpo-23225 "selectors: raise an exception if the selector is closed" which is a different approach (but it should also fix the reference cycle, I kept the "self._map = None" change). |
New changeset 1544bdc409be by Victor Stinner in branch '3.4': New changeset 6e7403bc906f by Victor Stinner in branch 'default': |
I closed the issue bpo-23225, so I'm also closing this issue. Thanks again Martin. |
I noticed that _ProactorBasePipeTransport doesn't clear its reference to the socket when it is closed. I changed this in the following commit (now merged in Python 3.4 & 3.5): |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: