This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: use AI_NUMERICHOST | AI_NUMERICSERV to skip getaddrinfo thread in asyncio
Type: enhancement Stage: patch review
Components: asyncio Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Ben.Darnell, asvetlov, graingert, yselivanov
Priority: normal Keywords: patch

Created on 2022-02-22 10:15 by graingert, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 31497 open graingert, 2022-02-22 10:23
Messages (5)
msg413699 - (view) Author: Thomas Grainger (graingert) * Date: 2022-02-22 10:15
now that the getaddrinfo lock has been removed on all platforms the numeric only host resolve in asyncio could be moved back into BaseEventLoop.getaddrinfo
msg413701 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-02-22 10:35
Could you provide more context for the proposed change?
msg413702 - (view) Author: Thomas Grainger (graingert) * Date: 2022-02-22 10:53
hello, it's actually a bit of a round about context, but it was brought up on a tornado issue where I was attempting to port the asyncio optimization to tornado: https://github.com/tornadoweb/tornado/issues/3113#issuecomment-1041019287

I think it would be better to use this AI_NUMERICHOST | AI_NUMERICSERV optimization from trio everywhere instead
msg415510 - (view) Author: Ben Darnell (Ben.Darnell) * Date: 2022-03-18 16:41
To summarize the justification, this patch does two things: it moves an optimization from create_connection to getaddrinfo, which makes it apply to more callers (including Tornado), and it makes the code simpler and less redundant (net reduction of 47 non-test lines in the patch). 

As far as we can tell, the reason it wasn't done this way in the first place is that at the time getaddrinfo held a global lock on some platforms, but this is no longer true. If there's still some locking in or around getaddrinfo on some platforms (or some libc implementations), this patch would be a bad idea. Is there a good way to test for that? I suppose we could set up a deliberately-slow DNS server and try to call getaddrinfo with AI_NUMERICHOST while another thread is blocked talking to that server, but that seems like a lot of test infrastructure to build out.
msg415511 - (view) Author: Ben Darnell (Ben.Darnell) * Date: 2022-03-18 16:45
On MacOS in 2015, getaddrinfo was found to be much slower than inet_pton. Unless that's changed, this patch would be a performance regression on that platform. Data and benchmark script in https://groups.google.com/g/python-tulip/c/-SFI8kkQEj4/m/m1-oCMSABgAJ
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 90980
2022-03-18 16:45:34Ben.Darnellsetmessages: + msg415511
2022-03-18 16:41:52Ben.Darnellsetnosy: + Ben.Darnell
messages: + msg415510
2022-02-22 10:53:58graingertsetmessages: + msg413702
2022-02-22 10:35:44asvetlovsetmessages: + msg413701
2022-02-22 10:23:16graingertsetkeywords: + patch
stage: patch review
pull_requests: + pull_request29627
2022-02-22 10:15:28graingertcreate