This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: TypeError in event loop finalizer, new in Python 3.4.3
Type: crash Stage:
Components: asyncio Versions: Python 3.4
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: chetan, gvanrossum, jab, oconnor663, vstinner, wjwwood, yselivanov, 山本泰宇
Priority: normal Keywords:

Created on 2015-02-28 16:13 by oconnor663, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (18)
msg236892 - (view) Author: Jack O'Connor (oconnor663) * Date: 2015-02-28 16:13
This toy program:

import asyncio
@asyncio.coroutine
def main():
    p = yield from asyncio.create_subprocess_shell('echo hi')
    yield from p.wait()
asyncio.get_event_loop().run_until_complete(main())

Produces this output on Arch Linux under Python 3.4.3 (but not 3.4.2):

hi
Exception ignored in: <bound method _UnixSelectorEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/base_events.py", line 361, in __del__
  File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/unix_events.py", line 57, in close
  File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/unix_events.py", line 138, in remove_signal_handler
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object
msg236894 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2015-02-28 16:36
Does it also have that error if you add `asyncio.get_event_loop().close()` to the end of the program?

Can you figure out what the value of `sig` and `handler` are in the traceback?

My gut feeling tells me this is due to `signal.default_int_handler` being None at that point in the program tear-down sequence. (Which is why I recommend closing the loop.)  The tear-down sequence at best has a partial order, which is why this may come and go depending on random variables like which Python version you use.
msg236904 - (view) Author: Jack O'Connor (oconnor663) * Date: 2015-02-28 19:36
`close()` fixes it; thanks for the workaround! When I throw a print statement inside `remove_signal_handler`, it says that sig is 17 and handler is 0. 17 looks to be SIGCHLD, presumably from the little echo subprocess exiting in this example.
msg236923 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2015-02-28 23:04
So this is still strange. When you do this, does it give the same exception?

>>> import signal
>>> signal.signal(signal.SIGCHLD, signal.SIG_DFL)
msg236960 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-01 13:26
I'm aware of the issue but I would prefer to not fix it.

The first problem is that you didn't close the event loop. I modified
recently the doc to explain at the begining that you should develop in the
debug mode. In this mode, you will see a warning if you don't close
explicitly the event loop.

The signal issue is that the event loop is closed very late during Python
finalization, and attributes of the signal module are already cleared.

Are you ok to not workaround this specific issue?

Closing an event loop implicitly late can lead to other various bugs.
msg237242 - (view) Author: Jack O'Connor (oconnor663) * Date: 2015-03-05 06:35
@gvanrossum, the last two lines you suggested don't give any error, as expected. Not sure why we're getting that error message in the toy example.

@haypo, closing the event loop explicitly works fine for me. But since you mentioned it, I don't see any warning when I leave out closing the event loop and turn on PYTHONASYNCIODEBUG=1. What should I be seeing?
msg237258 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-05 11:03
Did you read the latest doc? You should also use the -Wd command line
option to see resource warnings.
msg237259 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-05 11:06
Debug mode:
https://docs.python.org/dev/library/asyncio-dev.html#asyncio-debug-mode
msg237274 - (view) Author: Jack O'Connor (oconnor663) * Date: 2015-03-05 16:41
Got it, thanks for the heads up.
msg237286 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-05 20:54
I close the issue as wont fix. I may workaround the bug during Python finalization if more users report this issue.
msg239499 - (view) Author: Joshua Bronson (jab) * Date: 2015-03-29 16:55
Quoting Victor Stinner:
>  I may workaround the bug during Python finalization if more users report this issue.

Read the above so reporting I'm hitting this too fwiw.

Thanks for the great work on asyncio.
msg239500 - (view) Author: Joshua Bronson (jab) * Date: 2015-03-29 17:00
Not sure if it's related / helpful but just in case, since upgrading from 3.4.2 to 3.4.3, I'm now seeing this printed to stderr sometimes when my program exits:

Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in:
msg239524 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-29 21:33
> Exception ignored in: Exception ignored in: ...

Hum, I already saw such issues when exiting an asyncio application. It's probably because they are some pending tasks or transports which are never closed.

You can try to list tasks at exit using:

loop.close()
print(Task.all_tasks(loop=loop))
msg240248 - (view) Author: William Woodall (wjwwood) Date: 2015-04-08 04:05
I was getting the same error as the OP in my application.

I did something like this to work around the problem:

import asyncio
import atexit

def close_asyncio_loop():
    loop = None
    try:
        loop = asyncio.get_event_loop()
    except AttributeError:
        pass
    if loop is not None:
        loop.close()

atexit.register(close_asyncio_loop)

Is this an appropriate work around?

Why is it up to the application to close the loop explicitly?

Put another way, in what scenario would I want to close the loop outside of the application shutting down since it is irreversible?

Thanks in advance for your time.
msg248274 - (view) Author: Chetan Reddy (chetan) Date: 2015-08-08 15:24
I'm seeing the same exception as op with Python-3.5.0b4.

I'm writing a function in a library, and am using asyncio to make my function run faster. I'd like my library function to be useful even to users who aren't using asyncio and therefore won't call get_event_loop().close() at the end of their main function.

I can't call get_event_loop().close() in my own library function, because i don't want to be closing the event loop in case the user is using the event loop.

Is the recommended approach here for me to create my own loop and close it in my function (while saving and restoring the existing event loop in case the user has already created an event loop)? Guido's comment at http://bugs.python.org/msg205027 makes me think not.

The easiest solution might be to not require the user to call get_event_loop().close() . This will allow library writers to use asyncio.run_until_complete without worrying the user seeing an exception on exit. If you agree with this, I'm willing to spend the effort to track down this particular exception and provide a patch to fix it.
msg248286 - (view) Author: Chetan Reddy (chetan) Date: 2015-08-08 17:28
Disregard my previous comment. I realize now that I shouldn't be calling run_until_complete in a library function because it will fail if the user had already started running the event loop. I will attempt to use a create and use a new event loop in my library function. Apologies for adding noise to this page.
msg259205 - (view) Author: 山本泰宇 (山本泰宇) Date: 2016-01-29 12:00
> I may workaround the bug during Python finalization if more users report this issue.

+1 in my project:
https://github.com/cybozu/passh/issues/1

Anyway, I'd like to express my gratitude to asyncio.
msg320010 - (view) Author: William Woodall (wjwwood) Date: 2018-06-20 02:08
Just an update to my previous post. We ran into this issue again, but only noticed later because we do not see this problem on Ubuntu Bionic with Python 3.6.5, but we did see it again when we tested later on Ubuntu Xenial with Python 3.5.1.

See: https://github.com/ros2/launch/issues/84

So this seems to be fixed in later versions, unless it's just hidden now within the asyncio debug log messages (though I didn't see anything there either, at a glance).
History
Date User Action Args
2022-04-11 14:58:13adminsetgithub: 67736
2018-06-20 02:08:15wjwwoodsetmessages: + msg320010
2016-01-29 12:00:40山本泰宇setnosy: + 山本泰宇
messages: + msg259205
2015-08-08 17:28:55chetansetmessages: + msg248286
2015-08-08 15:24:46chetansetnosy: + chetan
messages: + msg248274
2015-04-08 04:05:32wjwwoodsetnosy: + wjwwood
messages: + msg240248
2015-03-29 21:33:51vstinnersetmessages: + msg239524
2015-03-29 17:00:10jabsetmessages: + msg239500
2015-03-29 16:55:15jabsetnosy: + jab
messages: + msg239499
2015-03-05 20:54:53vstinnersetstatus: open -> closed
resolution: wont fix
messages: + msg237286
2015-03-05 16:41:44oconnor663setmessages: + msg237274
2015-03-05 11:06:38vstinnersetmessages: + msg237259
2015-03-05 11:03:48vstinnersetmessages: + msg237258
2015-03-05 06:35:04oconnor663setmessages: + msg237242
2015-03-01 13:26:49vstinnersetmessages: + msg236960
2015-02-28 23:04:10gvanrossumsetmessages: + msg236923
2015-02-28 19:36:27oconnor663setmessages: + msg236904
2015-02-28 16:36:21gvanrossumsetmessages: + msg236894
2015-02-28 16:13:01oconnor663create