classification
Title: _signal module leak: test_interpreters leaked [1424, 1422, 1424] references
Type: Stage: patch review
Components: Interpreter Core Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: koubaa, shihai1991, vstinner
Priority: normal Keywords: patch

Created on 2020-09-04 09:53 by vstinner, last changed 2020-09-07 14:30 by vstinner.

Pull Requests
URL Status Linked Edit
PR 22087 merged vstinner, 2020-09-04 11:50
Messages (7)
msg376345 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-04 09:53
AMD64 Fedora Rawhide Refleaks 3.x:
https://buildbot.python.org/all/#/builders/565/builds/11

test_interpreters leaked [1424, 1422, 1424] references, sum=4270

According to git bisect, the leak was introduced by:

commit 71d1bd9569c8a497e279f2fea6fe47cd70a87ea3
Author: Mohamed Koubaa <koubaa.m@gmail.com>
Date:   Thu Sep 3 03:21:06 2020 -0500

    bpo-1635741: Port _signal module to multi-phase init (PEP 489) (GH-22049)

 .../2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst     |   1 +
 Modules/signalmodule.c                             | 168 +++++++++++----------
 2 files changed, 87 insertions(+), 82 deletions(-)


Example of leak:

$ ./python -m test -R 3:3 test_interpreters -m test.test_interpreters.TestInterpreterClose.test_from_current
0:00:00 load avg: 0.72 Run tests sequentially
0:00:00 load avg: 0.72 [1/1] test_interpreters
beginning 6 repetitions
123456
......
test_interpreters leaked [237, 237, 237] references, sum=711
test_interpreters leaked [18, 18, 18] memory blocks, sum=54
test_interpreters failed

== Tests result: FAILURE ==

1 test failed:
    test_interpreters

Total duration: 1.1 sec
Tests result: FAILURE
msg376348 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-04 10:03
The signal module is really special. In Python, only the main thread of the main interpreter is supposed to be allowed to run Python signal handlers (but the C signal handler can be executed by any thread).

The exec function of the _signal module runs actions each time a new instance of the _signal module is created, whereas some actions must only be done exactly once:

* modify Handlers[i].tripped
* modify Handlers[i].func
* Set SIGINT signal handler
* Initialize sigint_event event (Windows only): see bpo-41686

For example, calling signal.signal() in a subinterpreter raises ValueError("signal only works in main thread of the main interpreter").
msg376349 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-04 10:08
About the leak, the following three variables are also initialized multiple times by signal_exec():

static PyObject *DefaultHandler;
static PyObject *IgnoreHandler;
static PyObject *IntHandler;
msg376359 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-04 11:50
Another problem: PyOS_FiniInterrupts() is a public function of the C API which access global variables like IntHandler, DefaultHandler and IgnoreHandler.
msg376364 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-04 12:51
New changeset 4b8032e5a4994a7902076efa72fca1e2c85d8b7f by Victor Stinner in branch 'master':
bpo-41713: _signal doesn't use multi-phase init (GH-22087)
https://github.com/python/cpython/commit/4b8032e5a4994a7902076efa72fca1e2c85d8b7f
msg376469 - (view) Author: mohamed koubaa (koubaa) * Date: 2020-09-06 21:41
Sounds like there needs to be some python-wide global state that perhaps the signal module wraps rather than owns.  I could try to prototype it if you agree
msg376498 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-07 14:30
See also:

* bpo-40600: "Add an option to disallow creating more than one instance of a module".
* bpo-40288: [subinterpreters] atexit module should not be loaded more than once per interpreter
History
Date User Action Args
2020-09-07 14:30:08vstinnersetmessages: + msg376498
2020-09-06 21:41:40koubaasetnosy: + koubaa
messages: + msg376469
2020-09-04 14:05:37shihai1991setnosy: + shihai1991
2020-09-04 12:51:19vstinnersetmessages: + msg376364
2020-09-04 11:50:24vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request21173
2020-09-04 11:50:02vstinnersetmessages: + msg376359
2020-09-04 10:08:55vstinnersetmessages: + msg376349
2020-09-04 10:03:39vstinnersetmessages: + msg376348
2020-09-04 09:55:36vstinnersettitle: test_interpreters leaked [1424, 1422, 1424] references -> _signal module leak: test_interpreters leaked [1424, 1422, 1424] references
2020-09-04 09:53:41vstinnercreate