This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: NoneType object is not iterable error when asyncio Server.close() called
Type: behavior Stage: resolved
Components: asyncio Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, r.david.murray, vstinner, yselivanov
Priority: normal Keywords:

Created on 2014-10-06 22:22 by r.david.murray, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
waiter_bug.py r.david.murray, 2014-10-06 22:22
Messages (7)
msg228742 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-10-06 22:22
I'm writing a little web server using aiohttp.  I tried to write a unit test...since this is a client server situation I launched the asyncio program in a thread and did a urlopen from the main thread.  That all works fine...the problem is in the cleanup code.  When I try to stop the server I created via create_server, I get the error in the title.

I've attached a stripped down version of my attempted unit test.  It does require aiohttp...I'm not well enough versed in asyncio yet to rewrite it using more fundamental pieces.

I may be doing something stupid here, but it seems to me that even if I am close should not throw this error (the error implies that it is already shut down, so I'd think the close would be a no-op).
msg228745 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-10-06 23:23
I think that the main problem is that '_stop_server' is called from a main thread (by unittest machinery via addCleanup), whereas the loop is in the other thread. asyncio code is not thread-safe in general.

If I change your code slightly to avoid using addCleanup, then everything works: https://gist.github.com/1st1/8dd0a2d4aa1ffd895c52

FWIW I couldn't reproduce the "NoneType object is not iterable" error. On my machine with python3.4.1 it crashes with another error "AssertionError: server did not stop" on line 35.
msg228750 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-10-07 01:08
The 'Server did not stop' error was lying in wait for the NoneType bug to be fixed :)  So, if you didn't the NoneType, my test case isn't reproducing the problem for you, which is odd.  Using call_soon_threadsafe from an addCleanup in the _server method makes it work for me.  (Seems to work even without the threadsafe, but I think I'll keep it!)  Perhaps seeing the NoneType error is a timing issue, given that the problem is thread safety.

So, my apologies for the noise.  I guess I'm too used to threads, where it is typical to call the close or stop method from another thread.
msg228758 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-07 09:51
> Using call_soon_threadsafe from an addCleanup in the _server method makes it work for me. 

Maybe the documentation on thread safety should be improved? More warnings should be put in the documentation?

https://docs.python.org/dev/library/asyncio-dev.html#concurrency-and-multithreading
msg228760 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-07 09:54
Maybe we should more checks in debug mode in the Server class? For example, loop.call_soon() raises an AssertionError if it is called from the "wrong" thread" in debug mode.
msg228762 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-07 09:59
Oh, I didn't notice that waiter_bug.py does nothing :-) I added "unittest.main()". With PYTHONASYNCIODEBUG=1, I get *two* errors, maybe we don't need to add more checks in debug mode.

Again, maybe the debug mode should be better documented?
https://docs.python.org/dev/library/asyncio-dev.html#debug-mode-of-asyncio

Currently, it's at the end of the documentation.

ERROR:asyncio:<CoroWrapper Server.wait_closed() running at /home/haypo/prog/python/default/Lib/asyncio/base_events.py:143, created at waiter_bug.py:32> was never yielded from
Coroutine object created at (most recent call last):
  File "waiter_bug.py", line 43, in <module>
    unittest.main()
  File "/home/haypo/prog/python/default/Lib/unittest/main.py", line 93, in __init__
    self.runTests()
  File "/home/haypo/prog/python/default/Lib/unittest/main.py", line 244, in runTests
    self.result = testRunner.run(self.test)
  File "/home/haypo/prog/python/default/Lib/unittest/runner.py", line 168, in run
    test(result)
  File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/haypo/prog/python/default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/haypo/prog/python/default/Lib/unittest/case.py", line 625, in __call__
    return self.run(*args, **kwds)
  File "/home/haypo/prog/python/default/Lib/unittest/case.py", line 582, in run
    self.doCleanups()
  File "/home/haypo/prog/python/default/Lib/unittest/case.py", line 618, in doCleanups
    function(*args, **kwargs)
  File "waiter_bug.py", line 32, in _stop_server
    self.server.wait_closed()
DEBUG:asyncio:Using selector: EpollSelector
E
======================================================================
ERROR: test_version (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "waiter_bug.py", line 33, in _stop_server
    self.server_loop.stop()
  File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 285, in stop
    self.call_soon(_raise_stop_error)
  File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 373, in call_soon
    handle = self._call_soon(callback, args, check_loop=True)
  File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 382, in _call_soon
    self._assert_is_current_event_loop()
  File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 404, in _assert_is_current_event_loop
    "Non-thread-safe operation invoked on an event loop other "
RuntimeError: Non-thread-safe operation invoked on an event loop other than the current one
msg228765 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-10-07 11:47
Yes, documenting it at the beginning would be good.  I haven't gotten to the end of the docs yet, I like to experiment as I go along :)
History
Date User Action Args
2022-04-11 14:58:08adminsetgithub: 66762
2014-10-07 11:47:20r.david.murraysetmessages: + msg228765
2014-10-07 09:59:12vstinnersetmessages: + msg228762
2014-10-07 09:54:35vstinnersetmessages: + msg228760
2014-10-07 09:51:50vstinnersetmessages: + msg228758
2014-10-07 01:08:08r.david.murraysetstatus: open -> closed
resolution: not a bug
messages: + msg228750

stage: resolved
2014-10-06 23:23:13yselivanovsetmessages: + msg228745
2014-10-06 22:22:22r.david.murraycreate