This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Yaroslav.Halchenko
Recipients Yaroslav.Halchenko, asvetlov, yselivanov
Date 2020-05-15.15:30:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1589556602.37.0.773453748753.issue40634@roundup.psfhosted.org>
In-reply-to
Content
This is a reincarnation of previous issues such as 

- older https://bugs.python.org/issue21595 which partially (with ack on that) addressed the issue awhile back
- more recent https://bugs.python.org/issue38104 which was closed as "wont fix" since "the provided example finishes without any warning" on 3.8 (verified -- true for me with 3.8.3rc1); and with the explanation that "You spawn too many subprocesses that finish virtually at the same time. It leads to wakeup_fd overrun."
- additional similar reports could be found online, e.g. https://stackoverflow.com/a/52391791/1265472 .

In our project we are slowly introducing use of asyncio and have a mix of execution with asyncio and regular subprocess.Popen.  We do run lots of short lived processes serially, and I think it should be Ok, i.e. it should not cause underlying libraries to spit out some output to ignore unless we indeed just using them incorrectly somehow.

If we recreate the SelectorEventLoop for every separate execution via asyncio -- no ignored exception messages are displayed.  But if we start to reuse the same loop -- they eventually emerge.  If I enable asyncio debug and log it along with our own debug messages, the strange thing that they come around the points where we run using regular subprocess.Popen, not asyncio. See https://github.com/datalad/datalad/pull/4527#issuecomment-629289819 for more information.

Unfortunately I do not have (yet) any short script to reproduce it, but I would appreciate possible hints on how to figure out what is actually causing them in our particular case.  May be additional logging within asyncio could assist?
History
Date User Action Args
2020-05-15 15:30:02Yaroslav.Halchenkosetrecipients: + Yaroslav.Halchenko, asvetlov, yselivanov
2020-05-15 15:30:02Yaroslav.Halchenkosetmessageid: <1589556602.37.0.773453748753.issue40634@roundup.psfhosted.org>
2020-05-15 15:30:02Yaroslav.Halchenkolinkissue40634 messages
2020-05-15 15:30:01Yaroslav.Halchenkocreate