Author minrk
Recipients berker.peksag, methane, minrk, vstinner
Date 2016-09-14.13:47:11
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1473860831.57.0.402210718923.issue28147@psf.upfronthosting.co.za>
In-reply-to
Content
> dictresize() is called for converting split table to combined table.
> How is it triggered many times?

every `self.__dict__.pop` triggers a resize. According to https://www.python.org/dev/peps/pep-0412/#split-table-dictionaries `obj.__dict__` is always a split-table dict. I do not understand the dict implementation enough to say precisely why, but `pop` forces a recombine via `resize` because split-table dicts don't support deletion. In `dict_resize`, due to a `<=minused` condition, the size is guaranteed to at least double every time `dict_resize` is called. It would appear that after this, `__dict__` is again forced to be a split-table dict, though I'm not sure how or where this happens, but good old-fashioned printf debugging shows that `dict_resize` is called for every `__dict__.pop` because _PyDict_HasSplitTable is true every time pop is called.


> In your test code, which loop cause leak? new instance loop or re-use instance loop?

Both loops cause the leak. If the `pop_attr()` is not in `__init__`, then only the re-used instance has the leak. if `pop_attr` is in `__init__`, then it happens across instances as well. I will try to add more comments in the code to make this clearer.

Does anyone have a handy way to create a split-table dict other than on `obj.__dict__`?


> Please add an unit test which triggers the memory leak

I should not have used the term memory leak, and have updated the title to be more precise. It is not memory allocated without a corresponding free, instead it is unbounded growth of the memory owned by a split-table dict. Cleaning up the object does indeed clean up the memory associated with it. The included test exercises the bug with several iterations. Running the test several times with only one iteration would not exercise the bug.
History
Date User Action Args
2016-09-14 13:47:11minrksetrecipients: + minrk, vstinner, methane, berker.peksag
2016-09-14 13:47:11minrksetmessageid: <1473860831.57.0.402210718923.issue28147@psf.upfronthosting.co.za>
2016-09-14 13:47:11minrklinkissue28147 messages
2016-09-14 13:47:11minrkcreate