classification
Title: "signal only works in main thread" in main thread
Type: behavior Stage:
Components: Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Richard Warfield, eric.snow, vstinner
Priority: normal Keywords:

Created on 2019-11-24 06:06 by Richard Warfield, last changed 2019-12-17 17:14 by Richard Warfield.

Messages (6)
msg357390 - (view) Author: Richard Warfield (Richard Warfield) Date: 2019-11-24 06:06
I have an application (https://github.com/litxio/ptghci) using embedded Python, which needs to set a signal handler (and use the prompt-toolkit library which itself sets signal handlers).

My call to signal.signal is guarded by a check that we're running in the main thread:

        if threading.current_thread() is threading.main_thread():
            print (threading.current_thread().name)
            signal.signal(signal.SIGINT, int_handler)

And the above indeed prints "MainThread".  But this raises an exception:

  File "app.py", line 45, in __init__
    signal.signal(signal.SIGINT, int_handler)
  File "/usr/lib/python3.8/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread

This seems like something that should not happen.  

Now, I tried to generate a simple replicating example but thus far haven't been able to do so -- simply calling signal.signal from PyRun_SimpleString doesn't do the trick, even within a pthreads thread.
msg357391 - (view) Author: Richard Warfield (Richard Warfield) Date: 2019-11-24 06:10
I should mention, this behavior is new in 3.8.0.  It did not occur in 3.7.x.
msg358363 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-12-13 22:25
Before 3.8, the "signal" module checked against the thread in which the module was initially loaded, treating that thread as the "main" thread.  That same was true (and still is) for the "threading" module.  The problem for both modules is that the Python runtime may have actually been initialized in a different thread, which is the actual "main" thread.

Since Python 3.8 we store the ID of the thread where the runtime is initialized and use that in the check the "signal" module does.  However, the "threading" module still uses the ID of the thread where it is first imported.  So your check against "threading.main_thread()" must be in code that isn't running in the same thread where you ran Py_Initialize().  It probably used to work because you imported "signal" and "threading" for the first time in the same thread.

So what next?

First, I've created issue39042 to address the current different meanings of "main thread".  That should resolve the discrepancy between the signal and threading modules.

Second, what can we do to help embedders make sure they are running their code in the main thread (e.g. when setting signals)?  Is there any C-API we could add which would have helped you here?
msg358466 - (view) Author: Richard Warfield (Richard Warfield) Date: 2019-12-16 05:31
Thanks for looking into this.  Changing the behavior of the
"threading" module to be consistent with the runtime and "signal" module
would be sufficient, at least for my particular case.  If the "if
threading.current_thread() is threading.main_thread()" guard worked as
expected it would be clear how to resolve the problem.

On Sat, Dec 14, 2019 at 6:25 AM Eric Snow <report@bugs.python.org> wrote:

>
> Eric Snow <ericsnowcurrently@gmail.com> added the comment:
>
> Before 3.8, the "signal" module checked against the thread in which the
> module was initially loaded, treating that thread as the "main" thread.
> That same was true (and still is) for the "threading" module.  The problem
> for both modules is that the Python runtime may have actually been
> initialized in a different thread, which is the actual "main" thread.
>
> Since Python 3.8 we store the ID of the thread where the runtime is
> initialized and use that in the check the "signal" module does.  However,
> the "threading" module still uses the ID of the thread where it is first
> imported.  So your check against "threading.main_thread()" must be in code
> that isn't running in the same thread where you ran Py_Initialize().  It
> probably used to work because you imported "signal" and "threading" for the
> first time in the same thread.
>
> So what next?
>
> First, I've created issue39042 to address the current different meanings
> of "main thread".  That should resolve the discrepancy between the signal
> and threading modules.
>
> Second, what can we do to help embedders make sure they are running their
> code in the main thread (e.g. when setting signals)?  Is there any C-API we
> could add which would have helped you here?
>
> ----------
> nosy: +eric.snow, vstinner
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue38904>
> _______________________________________
>
msg358558 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-12-17 17:10
So resolving issue39042 would be enough, particularly if we backported
the change to 3.8?
msg358559 - (view) Author: Richard Warfield (Richard Warfield) Date: 2019-12-17 17:14
I think so, yes.

On Wed, Dec 18, 2019 at 1:10 AM Eric Snow <report@bugs.python.org> wrote:

>
> Eric Snow <ericsnowcurrently@gmail.com> added the comment:
>
> So resolving issue39042 would be enough, particularly if we backported
> the change to 3.8?
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue38904>
> _______________________________________
>
History
Date User Action Args
2019-12-17 17:14:27Richard Warfieldsetmessages: + msg358559
2019-12-17 17:10:52eric.snowsetmessages: + msg358558
2019-12-16 05:31:24Richard Warfieldsetmessages: + msg358466
2019-12-13 22:25:14eric.snowsetnosy: + vstinner, eric.snow
messages: + msg358363
2019-11-24 06:10:21Richard Warfieldsetmessages: + msg357391
2019-11-24 06:06:20Richard Warfieldcreate