classification
Title: await process.wait() does not work with a new_event_loop
Type: behavior Stage:
Components: asyncio Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Justin Mayfield, chetan, gvanrossum, vstinner, yselivanov
Priority: normal Keywords:

Created on 2015-08-10 09:30 by chetan, last changed 2016-09-04 23:04 by Justin Mayfield.

Files
File name Uploaded Description Edit
test.py chetan, 2015-08-10 09:30 simple program using a new_event_loop with subprocess_shell
Messages (4)
msg248353 - (view) Author: Chetan Reddy (chetan) Date: 2015-08-10 09:30
Attached test.py hangs with the following output when run with PYTHONASYNCIODEBUG1= and -Wdefault.

DEBUG:asyncio:Using selector: EpollSelector
DEBUG:asyncio:run shell command 'sleep 2'
DEBUG:asyncio:process 'sleep 2' created: pid 13801
INFO:asyncio:run shell command 'sleep 2': <_UnixSubprocessTransport pid=13801 running>

After i Ctrl-C it, i see the following traceback (paths edited)

^CTraceback (most recent call last):
  File "/chetan/test.py", line 17, in <module>
    main()
  File "/chetan/test.py", line 12, in main
    result =loop.run_until_complete(sleepWithShell(loop))
  File "/Python-3.5.0b4-clang/lib/python3.5/asyncio/base_events.py", line 329, in run_until_complete
    self.run_forever()
  File "/Python-3.5.0b4-clang/lib/python3.5/asyncio/base_events.py", line 300, in run_forever
    self._run_once()
  File "/Python-3.5.0b4-clang/lib/python3.5/asyncio/base_events.py", line 1142, in _run_once
    event_list = self._selector.select(timeout)
  File "/Python-3.5.0b4-clang/lib/python3.5/selectors.py", line 432, in select
    fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt
/Python-3.5.0b4-clang/lib/python3.5/asyncio/base_subprocess.py:117: ResourceWarning: unclosed transport <_UnixSubprocessTransport pid=13801 running>
/Python-3.5.0b4-clang/lib/python3.5/asyncio/base_events.py:384: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=True>
DEBUG:asyncio:Close <_UnixSelectorEventLoop running=False closed=False debug=True>
msg248354 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-08-10 09:48
It's better to use asyncio.get_event_loop() to create the event loop.
Otherwise, you have to create manually the child watcher:
https://docs.python.org/dev/library/asyncio-subprocess.html#subprocess-and-threads

If I recall correctly, you have to call asyncio. get_child_watcher().

What is your OS?

Maybe subprocess_exec/shell may ensure that a child watcher exist. Maybe
only in debug mode?
msg248358 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2015-08-10 13:05
I think it's unfortunate that this doesn't work. Note that the example program has no threads -- it just has a loop that isn't the default loop. The docs you refer to aren't very helpful.
msg274380 - (view) Author: Justin Mayfield (Justin Mayfield) * Date: 2016-09-04 23:04
I agree with Guido.  I spent a couple hours trying to debug some of my own code that turned out to be this issue.  My use case is single threaded too.

Perhaps I'm daft but I don't understand why the child watcher is part of the event loop policy.  At first glance it appears that it would be better suited being attached to any event loop in the main thread and then this wouldn't be an issue.

I wonder if this design is intended to support launching subprocesses from non-main thread event loops, which seems inherently dangerous and potentially misguided to me.
History
Date User Action Args
2016-09-04 23:04:57Justin Mayfieldsetnosy: + Justin Mayfield
messages: + msg274380
2015-08-10 13:05:22gvanrossumsetmessages: + msg248358
2015-08-10 09:48:44vstinnersetmessages: + msg248354
2015-08-10 09:30:38chetancreate