This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Does json.dumps have a memory leak?
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: cheryl.sabella, pitrou, rohandsa, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-11-16 08:20 by rohandsa, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (5)
msg306344 - (view) Author: Rohan D'Sa (rohandsa) Date: 2017-11-16 08:20
import gc, json

class leak(object):
    def __init__(self):
        pass

gc.set_debug(gc.DEBUG_LEAK)
while True:
    leak_ = leak()
    json.dumps(leak_.__dict__, indent=True)
    gc.collect()
    print(f"garbage count: {len(gc.garbage)}")

Using the following code under Python 3.6.3, the garbage count keeps increasing and windows task manager records steady memory increase.

However without indent json.dumps(self.__dict__), no leak is observed.
msg306345 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-11-16 09:01
indent=True just makes json to use Python implementation instead of C implementation. Python implementation uses closures which reference one other. Simple example not involving json is:

import gc

def f():
    def g():
        return h
    def h():
        return g
    return

gc.set_debug(gc.DEBUG_LEAK)
while True:
    f()
    gc.collect()
    print(f"garbage count: {len(gc.garbage)}")


The "leak" is caused by using gc.set_debug(gc.DEBUG_LEAK). gc.DEBUG_LEAK includes gc.DEBUG_COLLECTABLE, gc.DEBUG_UNCOLLECTABLE and gc.DEBUG_SAVEALL. gc.DEBUG_SAVEALL causes garbage-collected objects to be saved in gc.garbage for inspection. In normal circumstances they are collected.
msg306414 - (view) Author: Rohan D'Sa (rohandsa) Date: 2017-11-17 07:51
you are right. i realized later i actually had a leak in a com instantiated object, assumed it was a leak in the python and tried to find it using the gc module. 

The gc documentation led me down the garden path.

QUOTE

gc.garbage
A list of objects which the collector found to be unreachable but could not be freed (uncollectable objects). 

UNQUOTE

i assumed:
- cyclic references are unreachable but can be freed and hence collectable.
- __del__ finalizer (with cyclic references?) objects are unreachable and cannot be freed and hence uncollectable.
msg311290 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2018-01-30 19:31
Can this be closed as 'Not a Bug'?
msg311291 - (view) Author: Rohan D'Sa (rohandsa) Date: 2018-01-30 19:32
Yes. Thanks.

On 30 Jan 2018 8:31 PM, "Cheryl Sabella" <report@bugs.python.org> wrote:

>
> Cheryl Sabella <chekat2@gmail.com> added the comment:
>
> Can this be closed as 'Not a Bug'?
>
> ----------
> nosy: +csabella
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue32045>
> _______________________________________
>
History
Date User Action Args
2022-04-11 14:58:54adminsetgithub: 76226
2018-01-30 23:00:31cheryl.sabellasetstatus: open -> closed
resolution: not a bug
stage: resolved
2018-01-30 19:32:24rohandsasetmessages: + msg311291
2018-01-30 19:31:36cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg311290
2017-11-17 07:51:08rohandsasetmessages: + msg306414
2017-11-16 09:01:32serhiy.storchakasetnosy: + serhiy.storchaka, pitrou
messages: + msg306345
2017-11-16 08:20:19rohandsacreate