classification
Title: Memory leak when asyncio.open_connection raise
Type: resource usage Stage: patch review
Components: asyncio Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: aeros, asvetlov, bquinlan, jack__d, seer, whoKilledLora, yselivanov
Priority: normal Keywords: patch

Created on 2021-07-21 13:46 by seer, last changed 2021-08-05 20:05 by jack__d.

Messages (4)
msg397945 - (view) Author: Artem (seer) Date: 2021-07-21 13:46
I write some short example.

import resource
import asyncio


class B:
    def __init__(self, loop):
        self.loop = loop
        self.some_big_data = bytearray(1024 * 1024)  # 1Mb for memory bloating

    async def doStuff(self):
        if not await self.connect():
            return
        print('Stuff done')

    async def connect(self) -> bool:
        try:
            _, writer = await asyncio.open_connection('127.0.0.1', 12345, loop=self.loop)
            writer.close()
            return True
        except OSError as e:
            pass
        return False


class A:
    def __init__(self, loop):
        self.loop = loop

    async def doBStuff(self):
        b = B(self.loop)
        await b.doStuff()

    async def work(self):
        print('Working...')
        for _ in range(1000):
            await self.loop.create_task(self.doBStuff())
        print('Done.')
        print(
            'Memory usage {}kb'.format(
                resource.getrusage(
                    resource.RUSAGE_SELF).ru_maxrss))


async def amain(loop):
    a = A(loop)
    await a.work()


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(amain(loop))


100 cycles
"Memory usage 41980kb"

1000 cycles
"Memory usage 55412kb"

10000 cycles
"Memory usage 82880kb"

And so on...

Does anyone know workaround?
msg397982 - (view) Author: Andrey (whoKilledLora) Date: 2021-07-22 10:24
Confirmed. I have the same problem. I suspect this is related to https://bugs.python.org/issue41699.
msg398219 - (view) Author: Artem (seer) Date: 2021-07-26 10:09
Checked on 3.9.6 - still leaking.

Strange stuff, but if I write 

except OSError as e:
    del self

instead of 

except OSError as e:
    pass

leak is disappearing.
msg398234 - (view) Author: Kyle Stanley (aeros) * (Python committer) Date: 2021-07-26 15:46
Thank you Arteem, that should help indicate where the memory leak is present.
History
Date User Action Args
2021-08-05 20:05:54jack__dsetpull_requests: - pull_request26111
2021-08-05 20:03:43jack__dsetkeywords: + patch
nosy: + jack__d

pull_requests: + pull_request26111
stage: patch review
2021-07-26 15:46:34aerossetmessages: + msg398234
2021-07-26 10:09:39seersetmessages: + msg398219
versions: + Python 3.9, - Python 3.6
2021-07-22 13:48:35seersetnosy: + bquinlan
2021-07-22 10:24:17whoKilledLorasetnosy: + whoKilledLora
messages: + msg397982
2021-07-21 13:46:19seercreate