Author fried
Recipients asvetlov, fried, yselivanov
Date 2018-05-15.16:21:06
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1526401266.56.0.682650639539.issue33523@psf.upfronthosting.co.za>
In-reply-to
Content
At Facebook and Instagram we have large interconnected codebases without clear boundaries of ownership. As we move more and more services to utilize asyncio we are finding that once blocking (but fast) code paths, are now cropping up with asyncio code using run_until_complete().  Now this is fine for all the blocking callers, but some times we have async callers to that blocking code path and now it doesn't work.  

So we have two options revert the change to not use asyncio deep in the dep tree or Convert all functions in the stack to be asyncio.  Both are not possible and engineers have solved them in two crazy ways.

1. Nested Event Loops, when you hit a run_until_complete, create a new eventloop and do the async and return the result.
2. Like the first, but each library creates its own eventloop, and swaps it with the running loop for the duration of the run_until_complete, restoring the original loop when its done. 

Both of these ultimately have the same problem, everything on the primary event loop stops running until the new loop is complete. What if we could instead start servicing the existing eventloop from the run_until_complete. This would insure that tasks don't timeout.

This would allow us to convert things to asyncio faster without having to have absolute knowledge of a codebase and its call graph, and not have to have all engineers completely synchronized.
History
Date User Action Args
2018-05-15 16:21:06friedsetrecipients: + fried, asvetlov, yselivanov
2018-05-15 16:21:06friedsetmessageid: <1526401266.56.0.682650639539.issue33523@psf.upfronthosting.co.za>
2018-05-15 16:21:06friedlinkissue33523 messages
2018-05-15 16:21:06friedcreate