classification
Title: Python doesn't exit with proper resultcode on SIGINT in runpy (pymain_run_module)
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, graingert, gregory.p.smith, gvanrossum, lukasz.langa, ncoghlan
Priority: normal Keywords: patch

Created on 2020-08-20 19:58 by graingert, last changed 2020-10-05 16:10 by lukasz.langa. This issue is now closed.

Files
File name Uploaded Description Edit
test_exit.py graingert, 2020-08-20 19:58
test_exit.txt graingert, 2020-08-20 20:16
test_exit_runpy.py graingert, 2020-08-20 22:08
test_exit_command_stdin.py graingert, 2020-08-20 23:31
Pull Requests
URL Status Linked Edit
PR 21956 merged graingert, 2020-08-25 19:24
PR 22397 merged lukasz.langa, 2020-09-24 11:43
PR 22398 merged graingert, 2020-09-24 15:11
Messages (22)
msg375732 - (view) Author: Thomas Grainger (graingert) * Date: 2020-08-20 19:58
when running "python -m ham.py" KeyboardInterrupt should result in -2, but results in 1

see test case
msg375733 - (view) Author: Thomas Grainger (graingert) * Date: 2020-08-20 19:58
see also https://bugs.python.org/issue1054041
msg375735 - (view) Author: Thomas Grainger (graingert) * Date: 2020-08-20 22:08
I think I've eliminated runpy.py, as I still get a `-2` when using:

import runpy
runpy.run_module("ham")

or

runpy._run_module_as_main("ham")

see attached test_exit_runpy.py


It seems the only difference is if pymain_run_file or pymain_run_module is used.
msg375736 - (view) Author: Thomas Grainger (graingert) * Date: 2020-08-20 23:31
I've now eliminated pymain_run_stdin and pymain_run_command, both process a KeyboardInterrupt as -2

see attached test_exit_command_stdin.py
msg375775 - (view) Author: Thomas Grainger (graingert) * Date: 2020-08-21 22:50
adding vstinner, as this seems related to pymain_run_module rather than runpy
msg375898 - (view) Author: Thomas Grainger (graingert) * Date: 2020-08-25 20:29
I've confirmed that the pymain_run_module fails on master in the same way with Mac, Windows and Ubuntu
msg376984 - (view) Author: Thomas Grainger (graingert) * Date: 2020-09-16 10:33
adding `__main__` owners to nosy
msg377000 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-16 15:02
Thomas, can you explain the problem in English? Those elaborate test programs don’t do it for me, sorry.
msg377012 - (view) Author: Thomas Grainger (graingert) * Date: 2020-09-16 19:08
The return code of python on linux when the program is ended with a KeyboardInterrupt should be -2, this works in most cases - except when called via "python -m"


Does this repl help?

Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib, subprocess
>>> pathlib.Path("spam.py").write_text("raise KeyboardInterrupt")
23
>>> subprocess.run(["python3", "spam.py"])
Traceback (most recent call last):
  File "spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', 'spam.py'], returncode=-2)
>>> subprocess.run(["python3", "-m", "spam"])
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/graingert/projects/spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', '-m', 'spam'], returncode=1)
>>>
msg377013 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-16 19:34
Okay, I got it. Now my next question. What do you mean by "I've eliminated runpy" and various other remarks along those lines? Have you actually identified the root cause of the problem? What's your suggested fix (without test framework).
msg377014 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-09-16 19:36
> The return code of python on linux when the program is ended with 
> a KeyboardInterrupt should be -2

In general, the exit status for an unhandled KeyboardInterrupt (i.e. _Py_UnhandledKeyboardInterrupt) should be the same as the default SIGINT handler. This is implemented by exit_sigint in Modules/main.c. In POSIX, it uses the actual default SIGINT handler via kill(). In Windows, it uses the exit status that the default console control handler would use, STATUS_CONTROL_C_EXIT (0xC000_013A).
msg377019 - (view) Author: Thomas Grainger (graingert) * Date: 2020-09-16 19:41
> Okay, I got it. Now my next question. What do you mean by "I've eliminated runpy" and various other remarks along those lines?

I can use `runpy._run_module_as_main` from another Python entry, eg -c and get the correct `-2`

>>> subprocess.run(["python3", "-c", "__import__('runpy')._run_module_as_main('spam')"])
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/graingert/projects/osirium-main/spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', '-c', "__import__('runpy')._run_module_as_main('spam')"], returncode=-2)


> Have you actually identified the root cause of the problem? What's your suggested fix (without test framework).

No I don't have a fix
msg377020 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-16 19:44
Okay, so according to eryksun the fix requires C code. @graingert, are you interested in developing such a fix? Or @eryksun do you have an idea on how to fix it?
msg377021 - (view) Author: Thomas Grainger (graingert) * Date: 2020-09-16 20:16
I'm assuming _Py_UnhandledKeyboardInterrupt is getting incorrectly cleared somewhere in here

https://github.com/python/cpython/blob/fc23a9483ef0d7c98bea9f82392377d0b6ef7b18/Modules/main.c#L291-L300

    Py_DECREF(runpy);
    Py_DECREF(runmodule);
    Py_DECREF(module);
    Py_DECREF(runargs);
    if (result == NULL) {
        return pymain_exit_err_print();
    }
    Py_DECREF(result);
    return 0;
}
msg377022 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-16 20:40
Thanks, I think this is the fix:

diff --git a/Modules/main.c b/Modules/main.c
index 4a76f4461b..3d1bbee3a0 100644
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -288,6 +288,10 @@ pymain_run_module(const wchar_t *modname, int set_argv0)
         return pymain_exit_err_print();
     }
     result = PyObject_Call(runmodule, runargs, NULL);
+    if (!result && PyErr_Occurred() == PyExc_KeyboardInterrupt) {
+        _Py_UnhandledKeyboardInterrupt = 1;
+    }
+
     Py_DECREF(runpy);
     Py_DECREF(runmodule);
     Py_DECREF(module);


Can you submit it as a PR? I don't have the time (but I can review once you've got tests working etc.)
msg377024 - (view) Author: Thomas Grainger (graingert) * Date: 2020-09-16 21:08
I pushed that patch
msg377027 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-16 21:24
Why did you remove 3.10?
msg377028 - (view) Author: Thomas Grainger (graingert) * Date: 2020-09-16 21:24
nice the tests pass with that fix
msg377339 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-22 15:53
New changeset a68a2ad19c891faa891904b3da537911cc77df21 by Thomas Grainger in branch 'master':
bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (#21956)
https://github.com/python/cpython/commit/a68a2ad19c891faa891904b3da537911cc77df21
msg377447 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-09-24 14:34
New changeset ca8d46dd422e5aa10f444796d93faec5a8cbc1e0 by Łukasz Langa in branch '3.9':
[3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22397)
https://github.com/python/cpython/commit/ca8d46dd422e5aa10f444796d93faec5a8cbc1e0
msg377451 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-09-24 16:44
New changeset ae462297c08515f2c0dacf77b04816504142ba3e by Thomas Grainger in branch '3.8':
[3.8] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22398)
https://github.com/python/cpython/commit/ae462297c08515f2c0dacf77b04816504142ba3e
msg378057 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-10-05 16:10
New changeset c26a666e6751a9fad766ef83432b893f9b15ecaf by Łukasz Langa in branch '3.9':
[3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22397)
https://github.com/python/cpython/commit/c26a666e6751a9fad766ef83432b893f9b15ecaf
History
Date User Action Args
2020-10-05 16:10:20lukasz.langasetmessages: + msg378057
2020-09-24 16:44:22lukasz.langasetmessages: + msg377451
2020-09-24 15:11:17graingertsetpull_requests: + pull_request21438
2020-09-24 14:34:43lukasz.langasetmessages: + msg377447
2020-09-24 11:43:34lukasz.langasetnosy: + lukasz.langa

pull_requests: + pull_request21437
2020-09-24 11:36:35lukasz.langasetversions: + Python 3.10
2020-09-22 15:53:35gvanrossumsetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2020-09-22 15:53:11gvanrossumsetmessages: + msg377339
2020-09-16 21:24:43graingertsetmessages: + msg377028
2020-09-16 21:24:41gvanrossumsetmessages: + msg377027
2020-09-16 21:08:21graingertsetmessages: + msg377024
versions: - Python 3.10
2020-09-16 20:40:49gvanrossumsetstage: patch review -> needs patch
versions: + Python 3.10
2020-09-16 20:40:27gvanrossumsetmessages: + msg377022
2020-09-16 20:16:41graingertsetmessages: + msg377021
2020-09-16 19:44:20gvanrossumsetmessages: + msg377020
2020-09-16 19:41:45graingertsetmessages: + msg377019
2020-09-16 19:36:44eryksunsetnosy: + eryksun
messages: + msg377014
2020-09-16 19:34:32gvanrossumsetmessages: + msg377013
2020-09-16 19:08:56graingertsetmessages: + msg377012
2020-09-16 15:02:53gvanrossumsetmessages: + msg377000
2020-09-16 10:33:05graingertsetnosy: + gvanrossum
messages: + msg376984
2020-09-01 09:47:06vstinnersetnosy: - vstinner
2020-08-25 20:29:13graingertsetmessages: + msg375898
2020-08-25 19:24:33graingertsetkeywords: + patch
stage: patch review
pull_requests: + pull_request21065
2020-08-23 14:05:51graingertsettype: behavior
2020-08-21 22:50:11graingertsetnosy: + vstinner
messages: + msg375775
2020-08-20 23:31:18graingertsetfiles: + test_exit_command_stdin.py

messages: + msg375736
2020-08-20 22:13:33graingertsettitle: Python doesn't exit with proper resultcode on SIGINT in runpy -> Python doesn't exit with proper resultcode on SIGINT in runpy (pymain_run_module)
2020-08-20 22:08:44graingertsetfiles: + test_exit_runpy.py

messages: + msg375735
2020-08-20 20:16:49graingertsetfiles: + test_exit.txt
2020-08-20 20:12:31graingertsetnosy: + gregory.p.smith
2020-08-20 20:12:12graingertsetnosy: + ncoghlan
2020-08-20 19:58:51graingertsetmessages: + msg375733
2020-08-20 19:58:37graingertcreate