classification
Title: test__xxsubinterpreters leaked [1, 1, 1] references: test_ids_global()
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: phsilva, vstinner
Priority: normal Keywords: patch

Created on 2020-03-26 01:25 by vstinner, last changed 2020-04-01 15:09 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19151 phsilva, 2020-03-26 01:26
PR 19172 merged phsilva, 2020-03-26 07:11
PR 19273 merged phsilva, 2020-04-01 03:11
Messages (9)
msg365042 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-26 01:25
$ ./python -m test -R 3:3 test__xxsubinterpreters -m test_ids_global
0:00:00 load avg: 0.80 Run tests sequentially
0:00:00 load avg: 0.80 [1/1] test__xxsubinterpreters
beginning 6 repetitions
123456
......
test__xxsubinterpreters leaked [1, 1, 1] references, sum=3
test__xxsubinterpreters leaked [1, 1, 1] memory blocks, sum=3
test__xxsubinterpreters failed

== Tests result: FAILURE ==

1 test failed:
    test__xxsubinterpreters

Total duration: 819 ms
Tests result: FAILURE


It started to leak since:

commit 7dd549eb08939e1927fba818116f5202e76f8d73
Author: Paulo Henrique Silva <ph.silva@carta.com>
Date:   Tue Mar 24 23:19:58 2020 -0300

    bpo-1635741: Port _functools module to multiphase initialization (PEP 489) (GH-19151)
msg365044 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-26 01:36
The folowing test is enough to reproduce the leak:

    def test_ids_global(self):
        interp1 = interpreters.create()
        script, rpipe = _captured_script("pass")
        interpreters.run_string(interp1, script)
        rpipe.close()

        interp2 = interpreters.create()
        script, rpipe = _captured_script("pass")
        interpreters.run_string(interp2, script)
        rpipe.close()
msg365053 - (view) Author: Paulo Henrique Silva (phsilva) * Date: 2020-03-26 06:19
I've got it, will investigate asap.
msg365054 - (view) Author: Paulo Henrique Silva (phsilva) * Date: 2020-03-26 07:13
The module still uses static state. Fixed the leak and will convert it to use per-module state in a separate issue.
msg365066 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-26 12:47
New changeset b09ae3ff43111a336c0b706ea32fa07f88c992d9 by Paulo Henrique Silva in branch 'master':
bpo-40071: Fix refleak in _functools module (GH19172)
https://github.com/python/cpython/commit/b09ae3ff43111a336c0b706ea32fa07f88c992d9
msg365067 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-26 13:06
> (...) will convert it to use per-module state in a separate issue.

Great!
msg365446 - (view) Author: Paulo Henrique Silva (phsilva) * Date: 2020-04-01 03:15
As discussed on PR19172, this module uses a global state in functions that do not receive a PyModule* and right now converting such cases to per-module state is not trivial. I will wait for PEP-573 implementation that will hopefully make this easier.

Pushed PR19273 to avoid the potential crash induced by the original change.
msg365481 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-01 15:06
New changeset eacc07439591c97f69ab4a3d17391b009cd78ae2 by Paulo Henrique Silva in branch 'master':
bpo-40071: Fix potential crash in _functoolsmodule.c (GH-19273)
https://github.com/python/cpython/commit/eacc07439591c97f69ab4a3d17391b009cd78ae2
msg365483 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-01 15:09
I closed the issue, the leak is now fixed and _functools has been fixed.

I created bpo-40137: TODO list when PEP 573 "Module State Access from C Extension Methods" will be implemented.
History
Date User Action Args
2020-04-01 15:09:56vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg365483

stage: patch review -> resolved
2020-04-01 15:06:25vstinnersetmessages: + msg365481
2020-04-01 03:15:52phsilvasetmessages: + msg365446
2020-04-01 03:11:49phsilvasetpull_requests: + pull_request18628
2020-03-26 13:06:49vstinnersetmessages: + msg365067
2020-03-26 12:47:54vstinnersetmessages: + msg365066
2020-03-26 07:13:34phsilvasetmessages: + msg365054
2020-03-26 07:11:23phsilvasetpull_requests: + pull_request18532
2020-03-26 06:19:33phsilvasetmessages: + msg365053
2020-03-26 01:36:23vstinnersetmessages: + msg365044
2020-03-26 01:26:28phsilvasetkeywords: + patch
nosy: + phsilva

pull_requests: + pull_request18529
stage: patch review
2020-03-26 01:25:25vstinnercreate