classification
Title: asyncio: ugly error related to signal handlers at exit if the loop is not closed explicitly
Type: crash Stage: resolved
Components: asyncio Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: asvetlov Nosy List: Alex Brandt, asvetlov, gvanrossum, hlellelid, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2016-01-16 19:06 by Alex Brandt, last changed 2017-12-26 10:29 by asvetlov. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4956 merged asvetlov, 2017-12-21 06:38
PR 4962 merged python-dev, 2017-12-21 15:07
PR 5002 merged asvetlov, 2017-12-24 09:32
PR 5003 merged python-dev, 2017-12-24 11:50
PR 5010 merged asvetlov, 2017-12-25 21:21
PR 5014 merged python-dev, 2017-12-26 09:53
Messages (15)
msg258401 - (view) Author: Alex Brandt (Alex Brandt) Date: 2016-01-16 19:06
When using the suggested practice of setting a stop loop signal handler with:

loop.add_signal_handler(signal.SIGTERM, loop.stop)

The following stack trace is given when the signal runs:

ligament_1 | Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
ligament_1 | Traceback (most recent call last):
ligament_1 |   File "/usr/lib/python3.5/asyncio/base_events.py", line 387, in __del__
ligament_1 |   File "/usr/lib/python3.5/asyncio/unix_events.py", line 58, in close
ligament_1 |   File "/usr/lib/python3.5/asyncio/unix_events.py", line 139, in remove_signal_handler
ligament_1 |   File "/usr/lib/python3.5/signal.py", line 47, in signal
ligament_1 | TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

Since this happens during shutdown of the application I wouldn't consider this a high priority bug but it is quite annoying.  I've also not investigated if this interrupts the loop stopping procedure yet.
msg258402 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-01-16 19:44
Heh, this is weird. If the signal being removed is SIGTERM, the logic looks like it is definitely going to call signal.signal(signal.SIGTERM, signal.SIG_DFL). And it doesn't look like the signal module's globals have been eradicated yet (or you'd have gotten something like "TypeError: 'NoneType' object is not callable" instead).

You seem to be using Python 3.5.0.  Can you repro this with 3.5.1 or with the asyncio from github? Do you have a small self-contained program that repos it?
msg258429 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-01-16 22:06
The problem is that the signal module has been cleared when the destruction
has been called. You must not rely on destructors but call explicitly close
methods. Please run your app in asyncio debug mode and enable logs. See
asyncio doc.
msg258432 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-01-16 22:10
Victor, if the signal module has been cleared, how could it emit that error
message? signal.signal itself would no longer exist.

(I do agree with the solution you suggest -- though note that it may be
tricky to close the loop from inside the signal callback for SIGTERM, which
is presumably what is going on here.)
msg259592 - (view) Author: Hans Lellelid (hlellelid) Date: 2016-02-04 20:34
FWIW, I am experiencing the issue described here with Python 3.5.1.
msg308819 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-20 21:15
`remove_signal_handler()` should do nothing if `sys.is_finalizing()` is true.
msg308828 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2017-12-20 21:38
> `remove_signal_handler()` should do nothing if `sys.is_finalizing()` is true.

Probably a good idea.

See also https://github.com/python/asyncio/pull/456.
msg308830 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-20 22:15
> `remove_signal_handler()` should do nothing if `sys.is_finalizing()` is true.

I dislike this option.

If you want to use sys.is_finalizing(), I would prefer to modify _UnixSelectorEventLoop.close() to not try to remove signal handler if close() has been called too late during Python finalization. But I also expect a warning in this case, not hide bugs silently.

If you reach this case, close() has probably been called by BaseEventLoop.__del__().
msg308859 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-21 09:15
Implemented PR 4956 following Victor's suggestion.
msg308875 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-21 15:06
New changeset 4a02543cf97e8cbf9293741379f977b85531e4c2 by Andrew Svetlov in branch 'master':
bpo-26133: Dont unsubscribe signals in UNIX even loop on interpreter shutdown (#4956)
https://github.com/python/cpython/commit/4a02543cf97e8cbf9293741379f977b85531e4c2
msg308890 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-21 17:42
New changeset 3bc68cff5b821e83ee5df8b8cd13f4f54151b406 by Andrew Svetlov (Miss Islington (bot)) in branch '3.6':
bpo-26133: Dont unsubscribe signals in UNIX even loop on interpreter shutdown (GH-4956) (#4962)
https://github.com/python/cpython/commit/3bc68cff5b821e83ee5df8b8cd13f4f54151b406
msg308995 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-24 11:50
New changeset 4f146f9ed133b9ad56d4ee7a653396836af34067 by Andrew Svetlov in branch 'master':
bpo-26133: Clear signals list on interpreter finalizing (#5002)
https://github.com/python/cpython/commit/4f146f9ed133b9ad56d4ee7a653396836af34067
msg308996 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-24 12:31
New changeset 5ff5d1167de88eb37265dcaf1396d12617a0ace7 by Andrew Svetlov (Miss Islington (bot)) in branch '3.6':
bpo-26133: Clear signals list on interpreter finalizing (GH-5002) (#5003)
https://github.com/python/cpython/commit/5ff5d1167de88eb37265dcaf1396d12617a0ace7
msg309054 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-26 09:53
New changeset a8f4e15f3d33084862ddd3a7d58cd00034e94f16 by Andrew Svetlov in branch 'master':
bpo-26133: Fix typos (#5010)
https://github.com/python/cpython/commit/a8f4e15f3d33084862ddd3a7d58cd00034e94f16
msg309056 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-26 10:29
New changeset 32518b439b9590cce0ef0639e558dc1ce2e152bb by Andrew Svetlov (Miss Islington (bot)) in branch '3.6':
bpo-26133: Fix typos (GH-5010) (#5014)
https://github.com/python/cpython/commit/32518b439b9590cce0ef0639e558dc1ce2e152bb
History
Date User Action Args
2017-12-26 10:29:35asvetlovsetmessages: + msg309056
2017-12-26 09:53:58python-devsetpull_requests: + pull_request4904
2017-12-26 09:53:45asvetlovsetmessages: + msg309054
2017-12-25 21:21:31asvetlovsetpull_requests: + pull_request4899
2017-12-24 12:32:05asvetlovsetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.6, Python 3.7, - Python 3.5
2017-12-24 12:31:00asvetlovsetmessages: + msg308996
2017-12-24 11:50:17python-devsetpull_requests: + pull_request4892
2017-12-24 11:50:06asvetlovsetmessages: + msg308995
2017-12-24 09:32:48asvetlovsetpull_requests: + pull_request4891
2017-12-21 17:42:34asvetlovsetmessages: + msg308890
2017-12-21 15:07:38python-devsetpull_requests: + pull_request4853
2017-12-21 15:06:48asvetlovsetmessages: + msg308875
2017-12-21 09:15:26asvetlovsetmessages: + msg308859
2017-12-21 06:38:15asvetlovsetkeywords: + patch
stage: patch review
pull_requests: + pull_request4848
2017-12-20 22:15:34vstinnersetmessages: + msg308830
2017-12-20 21:38:56yselivanovsetmessages: + msg308828
2017-12-20 21:15:10asvetlovsetassignee: asvetlov

messages: + msg308819
nosy: + asvetlov
2016-02-04 20:34:37hlellelidsetnosy: + hlellelid
messages: + msg259592
2016-01-19 14:00:27vstinnersettitle: asyncio: ugly error related signal handler at exit if the loop is not closed explicitly -> asyncio: ugly error related to signal handlers at exit if the loop is not closed explicitly
2016-01-19 14:00:18vstinnersettitle: TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object in <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>> -> asyncio: ugly error related signal handler at exit if the loop is not closed explicitly
2016-01-16 22:10:49gvanrossumsetmessages: + msg258432
2016-01-16 22:06:12vstinnersetmessages: + msg258429
2016-01-16 19:44:28gvanrossumsetmessages: + msg258402
2016-01-16 19:06:06Alex Brandtcreate