Issue22572
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2014-10-06 22:22 by r.david.murray, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
waiter_bug.py | r.david.murray, 2014-10-06 22:22 |
Messages (7) | |||
---|---|---|---|
msg228742 - (view) | Author: R. David Murray (r.david.murray) * ![]() |
Date: 2014-10-06 22:22 | |
I'm writing a little web server using aiohttp. I tried to write a unit test...since this is a client server situation I launched the asyncio program in a thread and did a urlopen from the main thread. That all works fine...the problem is in the cleanup code. When I try to stop the server I created via create_server, I get the error in the title. I've attached a stripped down version of my attempted unit test. It does require aiohttp...I'm not well enough versed in asyncio yet to rewrite it using more fundamental pieces. I may be doing something stupid here, but it seems to me that even if I am close should not throw this error (the error implies that it is already shut down, so I'd think the close would be a no-op). |
|||
msg228745 - (view) | Author: Yury Selivanov (yselivanov) * ![]() |
Date: 2014-10-06 23:23 | |
I think that the main problem is that '_stop_server' is called from a main thread (by unittest machinery via addCleanup), whereas the loop is in the other thread. asyncio code is not thread-safe in general. If I change your code slightly to avoid using addCleanup, then everything works: https://gist.github.com/1st1/8dd0a2d4aa1ffd895c52 FWIW I couldn't reproduce the "NoneType object is not iterable" error. On my machine with python3.4.1 it crashes with another error "AssertionError: server did not stop" on line 35. |
|||
msg228750 - (view) | Author: R. David Murray (r.david.murray) * ![]() |
Date: 2014-10-07 01:08 | |
The 'Server did not stop' error was lying in wait for the NoneType bug to be fixed :) So, if you didn't the NoneType, my test case isn't reproducing the problem for you, which is odd. Using call_soon_threadsafe from an addCleanup in the _server method makes it work for me. (Seems to work even without the threadsafe, but I think I'll keep it!) Perhaps seeing the NoneType error is a timing issue, given that the problem is thread safety. So, my apologies for the noise. I guess I'm too used to threads, where it is typical to call the close or stop method from another thread. |
|||
msg228758 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2014-10-07 09:51 | |
> Using call_soon_threadsafe from an addCleanup in the _server method makes it work for me. Maybe the documentation on thread safety should be improved? More warnings should be put in the documentation? https://docs.python.org/dev/library/asyncio-dev.html#concurrency-and-multithreading |
|||
msg228760 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2014-10-07 09:54 | |
Maybe we should more checks in debug mode in the Server class? For example, loop.call_soon() raises an AssertionError if it is called from the "wrong" thread" in debug mode. |
|||
msg228762 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2014-10-07 09:59 | |
Oh, I didn't notice that waiter_bug.py does nothing :-) I added "unittest.main()". With PYTHONASYNCIODEBUG=1, I get *two* errors, maybe we don't need to add more checks in debug mode. Again, maybe the debug mode should be better documented? https://docs.python.org/dev/library/asyncio-dev.html#debug-mode-of-asyncio Currently, it's at the end of the documentation. ERROR:asyncio:<CoroWrapper Server.wait_closed() running at /home/haypo/prog/python/default/Lib/asyncio/base_events.py:143, created at waiter_bug.py:32> was never yielded from Coroutine object created at (most recent call last): File "waiter_bug.py", line 43, in <module> unittest.main() File "/home/haypo/prog/python/default/Lib/unittest/main.py", line 93, in __init__ self.runTests() File "/home/haypo/prog/python/default/Lib/unittest/main.py", line 244, in runTests self.result = testRunner.run(self.test) File "/home/haypo/prog/python/default/Lib/unittest/runner.py", line 168, in run test(result) File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 87, in __call__ return self.run(*args, **kwds) File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 125, in run test(result) File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 87, in __call__ return self.run(*args, **kwds) File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 125, in run test(result) File "/home/haypo/prog/python/default/Lib/unittest/case.py", line 625, in __call__ return self.run(*args, **kwds) File "/home/haypo/prog/python/default/Lib/unittest/case.py", line 582, in run self.doCleanups() File "/home/haypo/prog/python/default/Lib/unittest/case.py", line 618, in doCleanups function(*args, **kwargs) File "waiter_bug.py", line 32, in _stop_server self.server.wait_closed() DEBUG:asyncio:Using selector: EpollSelector E ====================================================================== ERROR: test_version (__main__.Test) ---------------------------------------------------------------------- Traceback (most recent call last): File "waiter_bug.py", line 33, in _stop_server self.server_loop.stop() File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 285, in stop self.call_soon(_raise_stop_error) File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 373, in call_soon handle = self._call_soon(callback, args, check_loop=True) File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 382, in _call_soon self._assert_is_current_event_loop() File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 404, in _assert_is_current_event_loop "Non-thread-safe operation invoked on an event loop other " RuntimeError: Non-thread-safe operation invoked on an event loop other than the current one |
|||
msg228765 - (view) | Author: R. David Murray (r.david.murray) * ![]() |
Date: 2014-10-07 11:47 | |
Yes, documenting it at the beginning would be good. I haven't gotten to the end of the docs yet, I like to experiment as I go along :) |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:08 | admin | set | github: 66762 |
2014-10-07 11:47:20 | r.david.murray | set | messages: + msg228765 |
2014-10-07 09:59:12 | vstinner | set | messages: + msg228762 |
2014-10-07 09:54:35 | vstinner | set | messages: + msg228760 |
2014-10-07 09:51:50 | vstinner | set | messages: + msg228758 |
2014-10-07 01:08:08 | r.david.murray | set | status: open -> closed resolution: not a bug messages: + msg228750 stage: resolved |
2014-10-06 23:23:13 | yselivanov | set | messages: + msg228745 |
2014-10-06 22:22:22 | r.david.murray | create |