when running "python -m ham.py" KeyboardInterrupt should result in -2, but results in 1

see test case

see also https://bugs.python.org/issue1054041

I think I've eliminated runpy.py, as I still get a `-2` when using:

import runpy
runpy.run_module("ham")

or

runpy._run_module_as_main("ham")

see attached test_exit_runpy.py


It seems the only difference is if pymain_run_file or pymain_run_module is used.

I've now eliminated pymain_run_stdin and pymain_run_command, both process a KeyboardInterrupt as -2

see attached test_exit_command_stdin.py

adding vstinner, as this seems related to pymain_run_module rather than runpy

I've confirmed that the pymain_run_module fails on master in the same way with Mac, Windows and Ubuntu

adding `__main__` owners to nosy

Thomas, can you explain the problem in English? Those elaborate test programs don’t do it for me, sorry.

The return code of python on linux when the program is ended with a KeyboardInterrupt should be -2, this works in most cases - except when called via "python -m"


Does this repl help?

Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib, subprocess
>>> pathlib.Path("spam.py").write_text("raise KeyboardInterrupt")
23
>>> subprocess.run(["python3", "spam.py"])
Traceback (most recent call last):
  File "spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', 'spam.py'], returncode=-2)
>>> subprocess.run(["python3", "-m", "spam"])
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/graingert/projects/spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', '-m', 'spam'], returncode=1)
>>>

Okay, I got it. Now my next question. What do you mean by "I've eliminated runpy" and various other remarks along those lines? Have you actually identified the root cause of the problem? What's your suggested fix (without test framework).

> The return code of python on linux when the program is ended with 
> a KeyboardInterrupt should be -2

In general, the exit status for an unhandled KeyboardInterrupt (i.e. _Py_UnhandledKeyboardInterrupt) should be the same as the default SIGINT handler. This is implemented by exit_sigint in Modules/main.c. In POSIX, it uses the actual default SIGINT handler via kill(). In Windows, it uses the exit status that the default console control handler would use, STATUS_CONTROL_C_EXIT (0xC000_013A).

> Okay, I got it. Now my next question. What do you mean by "I've eliminated runpy" and various other remarks along those lines?

I can use `runpy._run_module_as_main` from another Python entry, eg -c and get the correct `-2`

>>> subprocess.run(["python3", "-c", "__import__('runpy')._run_module_as_main('spam')"])
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/graingert/projects/osirium-main/spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', '-c', "__import__('runpy')._run_module_as_main('spam')"], returncode=-2)


> Have you actually identified the root cause of the problem? What's your suggested fix (without test framework).

No I don't have a fix

Okay, so according to eryksun the fix requires C code. @graingert, are you interested in developing such a fix? Or @eryksun do you have an idea on how to fix it?

I'm assuming _Py_UnhandledKeyboardInterrupt is getting incorrectly cleared somewhere in here

https://github.com/python/cpython/blob/fc23a9483ef0d7c98bea9f82392377d0b6ef7b18/Modules/main.c#L291-L300

    Py_DECREF(runpy);
    Py_DECREF(runmodule);
    Py_DECREF(module);
    Py_DECREF(runargs);
    if (result == NULL) {
        return pymain_exit_err_print();
    }
    Py_DECREF(result);
    return 0;
}

Thanks, I think this is the fix:

diff --git a/Modules/main.c b/Modules/main.c
index 4a76f4461b..3d1bbee3a0 100644
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -288,6 +288,10 @@ pymain_run_module(const wchar_t *modname, int set_argv0)
         return pymain_exit_err_print();
     }
     result = PyObject_Call(runmodule, runargs, NULL);
+    if (!result && PyErr_Occurred() == PyExc_KeyboardInterrupt) {
+        _Py_UnhandledKeyboardInterrupt = 1;
+    }
+
     Py_DECREF(runpy);
     Py_DECREF(runmodule);
     Py_DECREF(module);


Can you submit it as a PR? I don't have the time (but I can review once you've got tests working etc.)

I pushed that patch

Why did you remove 3.10?

nice the tests pass with that fix

New changeset a68a2ad19c891faa891904b3da537911cc77df21 by Thomas Grainger in branch 'master':
bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (#21956)
https://github.com/python/cpython/commit/a68a2ad19c891faa891904b3da537911cc77df21

New changeset ca8d46dd422e5aa10f444796d93faec5a8cbc1e0 by Łukasz Langa in branch '3.9':
[3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22397)
https://github.com/python/cpython/commit/ca8d46dd422e5aa10f444796d93faec5a8cbc1e0

New changeset ae462297c08515f2c0dacf77b04816504142ba3e by Thomas Grainger in branch '3.8':
[3.8] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22398)
https://github.com/python/cpython/commit/ae462297c08515f2c0dacf77b04816504142ba3e

New changeset c26a666e6751a9fad766ef83432b893f9b15ecaf by Łukasz Langa in branch '3.9':
[3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22397)
https://github.com/python/cpython/commit/c26a666e6751a9fad766ef83432b893f9b15ecaf