classification
Title: Unexpected behaviour NameError: name 'open' is not defined
Type: behavior Stage: resolved
Components: asyncio Versions: Python 3.9
process
Status: closed Resolution: duplicate
Dependencies: Superseder: logging: Trying to log during Python finalization with NameError: name 'open' is not defined
View: 26789
Assigned To: Nosy List: asvetlov, jryan, serhiy.storchaka, steven.daprano, vinay.sajip, vstinner, xtreak, yselivanov
Priority: normal Keywords:

Created on 2020-10-30 07:44 by jryan, last changed 2020-10-31 06:39 by vinay.sajip. This issue is now closed.

Messages (8)
msg379931 - (view) Author: john ryan (jryan) Date: 2020-10-30 07:44
My test environment runs Ubuntu 18.04 in a virtualbox hosted on Windows 8.1 and Python executes within a venv running Python 3.9.0 (Python 3.9.0 (default, Oct 26 2020, 09:02:51) 
[GCC 7.5.0] on linux

Running a test with unittest.IsolatedAsyncioTestCase my code hung so I hit ctrl-C twice to stop it and got the below traceback.

The really odd behaviour is that open is reported as not defined.

The behaviour is repeatable in my test, but I do not know how to produce a simple test case. I do not know what the issue is with my code.


 ^C^CTraceback (most recent call last):
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/async_case.py", line 158, in run
    return super().run(result)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/case.py", line 593, in run
    self._callTestMethod(testMethod)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/async_case.py", line 65, in _callTestMethod
    self._callMaybeAsync(method)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/async_case.py", line 88, in _callMaybeAsync
    return self._asyncioTestLoop.run_until_complete(fut)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
    self._run_once()
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/asyncio/base_events.py", line 1854, in _run_once
    event_list = self._selector.select(timeout)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/__main__.py", line 18, in <module>
    main(module=None)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/main.py", line 101, in __init__
    self.runTests()
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/main.py", line 271, in runTests
    self.result = testRunner.run(self.test)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/runner.py", line 176, in run
    test(result)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/suite.py", line 122, in run
    test(result)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/suite.py", line 122, in run
    test(result)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/suite.py", line 122, in run
    test(result)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/case.py", line 653, in __call__
    return self.run(*args, **kwds)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/async_case.py", line 160, in run
    self._tearDownAsyncioLoop()
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/unittest/async_case.py", line 126, in _tearDownAsyncioLoop
    loop.run_until_complete(self._asyncioCallsQueue.join())
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
    self._run_once()
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/asyncio/base_events.py", line 1854, in _run_once
    event_list = self._selector.select(timeout)
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt
Exception ignored in: <bound method BaseEventLoop.call_exception_handler of <_UnixSelectorEventLoop running=False closed=False debug=True>>
Traceback (most recent call last):
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/asyncio/base_events.py", line 1771, in call_exception_handler
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/logging/__init__.py", line 1463, in error
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/logging/__init__.py", line 1577, in _log
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/logging/__init__.py", line 1587, in handle
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/logging/__init__.py", line 1649, in callHandlers
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/logging/__init__.py", line 948, in handle
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/logging/__init__.py", line 1182, in emit
  File "/home/john/.pyenv/versions/3.9.0/lib/python3.9/logging/__init__.py", line 1171, in _open
NameError: name 'open' is not defined
msg379932 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-30 08:14
It can happen if the code is executed very late at the shutdown stage. The builtin module is restored to its initial state (open is imported from io) to get rid of reference cycles.
msg379934 - (view) Author: john ryan (jryan) Date: 2020-10-30 08:21
Thanks. That is understandable. I reported it in case it was helpful.
msg379935 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2020-10-30 08:28
See:

#26789 #39513

Serhiy, is a test case still needed? Should this be closed and pushed back to fixing logging?
msg379953 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-30 12:03
Yes, solving any of issue26789 or issue39513 will likely solve also this issue. But this issue is the one which relates to asyncio tests, so there may be some flaw in tests which should be fixed too. Solving issue26789 or issue39513 will eliminate symptoms, so it would be harder to fix the bug in test is there is one.
msg379954 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2020-10-30 12:39
This is a duplicate of below

https://bugs.python.org/issue26789
https://bugs.python.org/issue39513
msg379964 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-30 14:23
I mark this issue as a duplicate of bpo-26789.
msg380044 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2020-10-31 06:39
> Should this be closed and pushed back to fixing logging?

It's not particularly a logging issue, though it can surface in logging because that's a mechanism to notify users about stuff. After all, most parts of the stdlib would expect open() to always be available. ISTM it's more a case of shutdown behaviour and user expectations relating to that. Users should be able to avoid any logging by calling logging.shutdown(), which can be done during a controlled process shutdown in user code.
History
Date User Action Args
2020-10-31 06:39:05vinay.sajipsetmessages: + msg380044
2020-10-30 14:23:31vstinnersetstatus: open -> closed

superseder: logging: Trying to log during Python finalization with NameError: name 'open' is not defined

nosy: + vstinner
messages: + msg379964
resolution: duplicate
stage: resolved
2020-10-30 12:39:00xtreaksetnosy: + xtreak
messages: + msg379954
2020-10-30 12:03:45serhiy.storchakasetmessages: + msg379953
2020-10-30 08:28:08steven.dapranosetnosy: + steven.daprano
messages: + msg379935
2020-10-30 08:21:15jryansetmessages: + msg379934
2020-10-30 08:14:07serhiy.storchakasetnosy: + vinay.sajip, serhiy.storchaka
messages: + msg379932
2020-10-30 07:44:09jryancreate