classification
Title: Reference leak regression with Python3.8a3
Type: resource usage Stage: resolved
Components: Versions: Python 3.8
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Frame stack is not cleaned after execution is finished with return
View: 36851
Assigned To: Nosy List: kayhayen, nanjekyejoannah, vstinner
Priority: normal Keywords: patch

Created on 2019-04-25 20:28 by kayhayen, last changed 2019-05-13 17:17 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12960 merged vstinner, 2019-04-26 00:33
PR 12961 merged vstinner, 2019-04-26 02:10
PR 13231 merged vstinner, 2019-05-13 17:17
Messages (10)
msg340861 - (view) Author: Kay Hayen (kayhayen) Date: 2019-04-25 20:28
Much like #9366 the same file can be used. This reference leaks according to Nuitka comparative testing:

simpleFunction59: FAILED 129511 129512 leaked 1

def simpleFunction59():
    a = 3
    b = 5

    try:
        a = a * 2

        return a
    finally:
        return a / b


I would be guessing, that you are leaking the return value when returning again.
msg340876 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-26 02:09
New changeset 4d29983185bc12ca685a1eb3873bacb8a7b67416 by Victor Stinner in branch 'master':
bpo-36725: regrtest: add TestResult type (GH-12960)
https://github.com/python/cpython/commit/4d29983185bc12ca685a1eb3873bacb8a7b67416
msg340883 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-26 06:40
New changeset 3cde440f20a9db75fb2c4e65e8e4d04a53216a2d by Victor Stinner in branch 'master':
bpo-36725: Refactor regrtest multiprocessing code (GH-12961)
https://github.com/python/cpython/commit/3cde440f20a9db75fb2c4e65e8e4d04a53216a2d
msg340888 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-26 07:33
Sorry, the two commits are related to bpo-36719 not to this issue.
msg340890 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-26 07:40
I wrote an unit test to be able to re-use regrtest memory leak hunter:

import unittest

def simpleFunction59():
    a = 1
    try:
        return a
    finally:
        return a

class Tests(unittest.TestCase):
    def test_bug(self):
        for _ in range(10):
            simpleFunction59()


I confirm that there is a leak:

$ ./python -m test -R 3:3 test_bug
...
test_bug leaked [10, 10, 10] references, sum=30
...
msg340901 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-26 10:16
New changeset 1069d38fa18f3a4f97c2e358bcb3b82cab1c051b by Victor Stinner in branch '3.7':
[3.7] bpo-36719: sync regrtest with master branch (GH-12967)
https://github.com/python/cpython/commit/1069d38fa18f3a4f97c2e358bcb3b82cab1c051b
msg340917 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python triager) Date: 2019-04-26 14:01
Let me work on this.
msg340920 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python triager) Date: 2019-04-26 14:47
@ victor, I have tried your example but I can not reproduce the leak as you did above. May be am missing something?

I used some other tool and got this summary when I was investigating this:

                            types |   # objects |   total size
================================= | =========== | ============
                      <class 'str |        3916 |    361.40 KB
                     <class 'list |        2513 |    237.41 KB
                     <class 'dict |         197 |    120.28 KB
                     <class 'code |         617 |     86.79 KB
                     <class 'type |          81 |     80.71 KB
                      <class 'int |         497 |     13.60 KB
                    <class 'tuple |         199 |     12.80 KB
              function (__init__) |          61 |      8.10 KB
                  <class 'weakref |          87 |      6.80 KB
        <class 'getset_descriptor |          67 |      4.71 KB
  <class 'collections.OrderedDict |           3 |      4.22 KB
            <class 'enum.EnumMeta |           3 |      2.98 KB
              function (__call__) |          14 |      1.86 KB
                     <class 'cell |          37 |      1.73 KB
                   <enum 'Signals |          32 |      1.62 KB

Am still investigating though.
msg340921 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python triager) Date: 2019-04-26 15:12
I am able to reproduce it finally.

Run tests sequentially
0:00:00 load avg: 0.21 [1/1] test_bug
beginning 6 repetitions
123456
......
test_bug leaked [10, 10, 10] references, sum=30
test_bug failed

== Tests result: FAILURE ==

1 test failed:
    test_bug

Total duration: 220 ms
Tests result: FAILURE
msg342340 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-13 15:31
This leak has been fixed by:

commit f00828a742d2e88c910bdfd00f08fcd998554ba5 (refs/bisect/bad)
Author: Pablo Galindo <Pablogsal@gmail.com>
Date:   Thu May 9 16:52:02 2019 +0100

    bpo-36851: Clean the frame stack if the execution ends with a return and the stack is not empty (GH-13191)
History
Date User Action Args
2019-05-13 17:17:56vstinnersetpull_requests: + pull_request13201
2019-05-13 15:31:06vstinnersetstatus: open -> closed
superseder: Frame stack is not cleaned after execution is finished with return
messages: + msg342340

resolution: duplicate
stage: patch review -> resolved
2019-05-13 14:13:57nanjekyejoannahsetassignee: nanjekyejoannah ->
2019-05-13 13:58:59vstinnersetpull_requests: - pull_request13188
2019-05-13 13:51:31vstinnersetstage: needs patch -> patch review
pull_requests: + pull_request13188
2019-05-12 05:33:13xtreaksetstage: patch review -> needs patch
2019-05-12 05:32:41xtreaksetpull_requests: - pull_request13139
2019-05-09 23:55:36vstinnersetpull_requests: + pull_request13139
2019-04-26 15:12:43nanjekyejoannahsetmessages: + msg340921
2019-04-26 14:47:19nanjekyejoannahsetmessages: + msg340920
2019-04-26 14:02:00nanjekyejoannahsetassignee: nanjekyejoannah
2019-04-26 14:01:47nanjekyejoannahsetnosy: + nanjekyejoannah
messages: + msg340917
2019-04-26 10:16:34vstinnersetmessages: + msg340901
2019-04-26 07:40:30vstinnersetmessages: + msg340890
2019-04-26 07:33:33vstinnersetmessages: + msg340888
2019-04-26 06:40:44vstinnersetmessages: + msg340883
2019-04-26 02:10:58vstinnersetpull_requests: + pull_request12887
2019-04-26 02:09:14vstinnersetmessages: + msg340876
2019-04-26 00:33:05vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request12886
2019-04-26 00:32:55vstinnersetnosy: + vstinner
2019-04-25 20:28:27kayhayencreate