This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: AST change introduced tons of reference leaks
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: dino.viehland Nosy List: dino.viehland, eelizondo, vstinner
Priority: normal Keywords: patch

Created on 2019-09-13 09:37 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16127 merged eelizondo, 2019-09-13 18:54
Messages (4)
msg352258 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-13 09:37
The following change introduced tons of reference leaks:

commit ac46eb4ad6662cf6d771b20d8963658b2186c48c
Author: Dino Viehland <dinoviehland@fb.com>
Date:   Wed Sep 11 10:16:34 2019 -0700

    bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)
    
    Summary: This mostly migrates Python-ast.c to PEP384 and removes all statics from the whole file. This modifies the generator itself that generates the Python-ast.c. It leaves in the usage of _PyObject_LookupAttr even though it's not fully PEP384 compatible (this could always be shimmed in by anyone who needs it).


Full list of tests which leaked in build:
https://buildbot.python.org/all/#/builders/1/builds/712

test_ast
test_asyncio
test_builtin
test_capi
test_check_c_globals
test_clinic
test_compile
test_dataclasses
test_dbm
test_dbm_dumb
test_decimal
test_fstring
test_getpass
test_idle
test_importlib
test_inspect
test_pydoc
test_source_encoding
test_symtable
test_sys
test_type_comments
test_types
test_unittest


vstinner@apu$ ./python -m test -j0 -R 3:3 --fromfile=leak
(...)
test_dbm_dumb leaked [938, 896, 908] references, sum=2742
test_dbm leaked [144, 144, 144] references, sum=432
test_builtin leaked [128, 128, 128] references, sum=384
test_clinic leaked [1935, 1933, 1935] references, sum=5803
test_getpass leaked [21, 21, 21] references, sum=63
test_dataclasses leaked [8, 8, 8] references, sum=24
test_idle leaked [69, 69, 69] references, sum=207
test_inspect leaked [539, 539, 539] references, sum=1617
test_fstring leaked [104, 104, 104] references, sum=312
test_type_comments leaked [1337, 1335, 1337] references, sum=4009
test_types leaked [7, 7, 7] references, sum=21
test_compile leaked [3690, 3688, 3690] references, sum=11068
test_sys leaked [1, 1, 1] references, sum=3
test_pydoc leaked [1504, 1502, 1504] references, sum=4510
test_ast leaked [357792, 357790, 357792] references, sum=1073374
test_ast leaked [12, 11, 12] memory blocks, sum=35
test_unittest leaked [3803, 3801, 3803] references, sum=11407
test_decimal leaked [841, 841, 841] references, sum=2523
test_capi leaked [2, 2, 2] references, sum=6

I interrupted test_asyncio test: it's too slow.


Reverting the commit ac46eb4ad6662cf6d771b20d8963658b2186c48c fix most reference leaks, except of test_capi: bpo-38150.
msg352388 - (view) Author: Eddie Elizondo (eelizondo) * Date: 2019-09-13 18:37
I have a fix for this coming up.
msg352441 - (view) Author: Eddie Elizondo (eelizondo) * Date: 2019-09-14 16:23
The PR has been merged so the issue can be closed now
msg352526 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-16 07:57
commit 0247e80f3c529900689425676342cb70ea31a13d
Author: Eddie Elizondo <eelizondo@fb.com>
Date:   Sat Sep 14 09:38:17 2019 -0400

    Fix leaks in Python-ast.c (#16127)
History
Date User Action Args
2022-04-11 14:59:20adminsetgithub: 82333
2019-09-16 07:57:11vstinnersetmessages: + msg352526
2019-09-14 16:46:31ammar2setstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-09-14 16:23:08eelizondosetmessages: + msg352441
2019-09-13 18:54:51eelizondosetkeywords: + patch
stage: patch review
pull_requests: + pull_request15738
2019-09-13 18:37:05eelizondosetnosy: + eelizondo
messages: + msg352388
2019-09-13 12:04:12dino.viehlandsetassignee: dino.viehland

nosy: + dino.viehland
2019-09-13 09:38:56vstinnerlinkissue38151 superseder
2019-09-13 09:37:08vstinnercreate