Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reference leak regression with Python3.8a3 #80906

Closed
kayhayen mannequin opened this issue Apr 25, 2019 · 10 comments
Closed

Reference leak regression with Python3.8a3 #80906

kayhayen mannequin opened this issue Apr 25, 2019 · 10 comments
Labels
3.8 only security fixes performance Performance or resource usage

Comments

@kayhayen
Copy link
Mannequin

kayhayen mannequin commented Apr 25, 2019

BPO 36725
Nosy @vstinner, @nanjekyejoannah
PRs
  • bpo-36725: regrtest: add TestResult type #12960
  • bpo-36725: Refactor regrtest multiprocessing code #12961
  • bpo-36719: regrtest -jN no longer stops on crash #13231
  • Superseder
  • bpo-36851: Frame stack is not cleaned after execution is finished with return
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-05-13.15:31:06.436>
    created_at = <Date 2019-04-25.20:28:27.757>
    labels = ['3.8', 'performance']
    title = 'Reference leak regression with Python3.8a3'
    updated_at = <Date 2019-05-13.17:17:56.569>
    user = 'https://bugs.python.org/kayhayen'

    bugs.python.org fields:

    activity = <Date 2019-05-13.17:17:56.569>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-05-13.15:31:06.436>
    closer = 'vstinner'
    components = []
    creation = <Date 2019-04-25.20:28:27.757>
    creator = 'kayhayen'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36725
    keywords = ['patch']
    message_count = 10.0
    messages = ['340861', '340876', '340883', '340888', '340890', '340901', '340917', '340920', '340921', '342340']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'kayhayen', 'nanjekyejoannah']
    pr_nums = ['12960', '12961', '13231']
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '36851'
    type = 'resource usage'
    url = 'https://bugs.python.org/issue36725'
    versions = ['Python 3.8']

    @kayhayen
    Copy link
    Mannequin Author

    kayhayen mannequin commented Apr 25, 2019

    Much like bpo-9366 the same file can be used. This reference leaks according to Nuitka comparative testing:

    simpleFunction59: FAILED 129511 129512 leaked 1

    def simpleFunction59():
        a = 3
        b = 5
    try:
        a = a * 2
    
            return a
        finally:
            return a / b

    I would be guessing, that you are leaking the return value when returning again.

    @kayhayen kayhayen mannequin added 3.8 only security fixes performance Performance or resource usage labels Apr 25, 2019
    @vstinner
    Copy link
    Member

    New changeset 4d29983 by Victor Stinner in branch 'master':
    bpo-36725: regrtest: add TestResult type (GH-12960)
    4d29983

    @vstinner
    Copy link
    Member

    New changeset 3cde440 by Victor Stinner in branch 'master':
    bpo-36725: Refactor regrtest multiprocessing code (GH-12961)
    3cde440

    @vstinner
    Copy link
    Member

    Sorry, the two commits are related to bpo-36719 not to this issue.

    @vstinner
    Copy link
    Member

    I wrote an unit test to be able to re-use regrtest memory leak hunter:

    import unittest
    
    def simpleFunction59():
        a = 1
        try:
            return a
        finally:
            return a
    
    class Tests(unittest.TestCase):
        def test_bug(self):
            for _ in range(10):
                simpleFunction59()

    I confirm that there is a leak:

    $ ./python -m test -R 3:3 test_bug
    ...
    test_bug leaked [10, 10, 10] references, sum=30
    ...

    @vstinner
    Copy link
    Member

    New changeset 1069d38 by Victor Stinner in branch '3.7':
    [3.7] bpo-36719: sync regrtest with master branch (GH-12967)
    1069d38

    @nanjekyejoannah
    Copy link
    Member

    Let me work on this.

    @nanjekyejoannah nanjekyejoannah self-assigned this Apr 26, 2019
    @nanjekyejoannah
    Copy link
    Member

    @ victor, I have tried your example but I can not reproduce the leak as you did above. May be am missing something?

    I used some other tool and got this summary when I was investigating this:

                            types |   # objects |   total size
    

    ================================= | =========== | ============
    <class 'str | 3916 | 361.40 KB
    <class 'list | 2513 | 237.41 KB
    <class 'dict | 197 | 120.28 KB
    <class 'code | 617 | 86.79 KB
    <class 'type | 81 | 80.71 KB
    <class 'int | 497 | 13.60 KB
    <class 'tuple | 199 | 12.80 KB
    function (init) | 61 | 8.10 KB
    <class 'weakref | 87 | 6.80 KB
    <class 'getset_descriptor | 67 | 4.71 KB
    <class 'collections.OrderedDict | 3 | 4.22 KB
    <class 'enum.EnumMeta | 3 | 2.98 KB
    function (call) | 14 | 1.86 KB
    <class 'cell | 37 | 1.73 KB
    <enum 'Signals | 32 | 1.62 KB

    Am still investigating though.

    @nanjekyejoannah
    Copy link
    Member

    I am able to reproduce it finally.

    Run tests sequentially
    0:00:00 load avg: 0.21 [1/1] test_bug
    beginning 6 repetitions
    123456
    ......
    test_bug leaked [10, 10, 10] references, sum=30
    test_bug failed

    == Tests result: FAILURE ==

    1 test failed:
    test_bug

    Total duration: 220 ms
    Tests result: FAILURE

    @nanjekyejoannah nanjekyejoannah removed their assignment May 13, 2019
    @vstinner
    Copy link
    Member

    This leak has been fixed by:

    commit f00828a (refs/bisect/bad)
    Author: Pablo Galindo <Pablogsal@gmail.com>
    Date: Thu May 9 16:52:02 2019 +0100

    bpo-36851: Clean the frame stack if the execution ends with a return and the stack is not empty (GH-13191)
    

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes performance Performance or resource usage
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants