This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test__xxsubinterpreters leaked [1486, 1484, 1484, 1484] references
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, pablogsal, vstinner
Priority: normal Keywords:

Created on 2020-05-08 03:04 by pablogsal, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (4)
msg368411 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-05-08 03:04
Minimal reproducer:

./python -m test test__xxsubinterpreters -m test_custom___reduce__ -R :
0:00:00 load avg: 3.20 Run tests sequentially
0:00:00 load avg: 3.20 [1/1] test__xxsubinterpreters
beginning 9 repetitions
123456789
.........
test__xxsubinterpreters leaked [1486, 1484, 1484, 1484] references, sum=5938
test__xxsubinterpreters leaked [399, 398, 398, 398] memory blocks, sum=1593
test__xxsubinterpreters failed

== Tests result: FAILURE ==

1 test failed:
    test__xxsubinterpreters

Total duration: 2.2 sec

Bisecting points to:

commit a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57
Author: Eric Snow <ericsnowcurrently@gmail.com>
Date:   Thu May 7 08:56:01 2020 -0600

    bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)

    (Note: PEP 554 is not accepted and the implementation in the code base is a private one for use in the test suite.)

    If code running in a subinterpreter raises an uncaught exception then the "run" call in the calling interpreter fails. A RunFailedError is raised there that summarizes the original exception as a string. The actual exception type, __cause__, __context__, state, etc. are all discarded. This turned out to be functionally insufficient in practice. There is a more helpful solution (and PEP 554 has been updated appropriately).

    This change adds the exception propagation behavior described in PEP 554 to the _xxsubinterpreters module. With this change a copy of the original exception is set to __cause__ on the RunFailedError. For now we are using "pickle", which preserves the exception's state. We also preserve the original __cause__, __context__, and __traceback__ (since "pickle" does not preserve those).

    https://bugs.python.org/issue32604

 Lib/test/test__xxsubinterpreters.py |  301 ++++++++-
 Modules/_xxsubinterpretersmodule.c  | 1139 +++++++++++++++++++++++++++++++----
 2 files changed, 1317 insertions(+), 123 deletions(-)
bisect run success
msg368581 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-10 10:01
FYI I also reported the issue to https://bugs.python.org/issue32604#msg368395 which introduched the regression. But you can keep this issue open until it's fixed.
msg368847 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-14 16:20
I wrote PR 20089 to revert the change which introduced the leak.
msg368850 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-14 16:49
The revert fixed the test:

$ ./python -m test -R 3:3 test__xxsubinterpreters 
(...)
Tests result: SUCCESS
History
Date User Action Args
2022-04-11 14:59:30adminsetgithub: 84736
2020-05-14 16:49:06vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg368850

stage: resolved
2020-05-14 16:20:03vstinnersetmessages: + msg368847
2020-05-10 10:01:06vstinnersetmessages: + msg368581
2020-05-08 03:04:25pablogsalcreate