This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author tim.peters
Recipients ian_osh, tim.peters
Date 2020-07-27.23:59:15
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1595894355.79.0.304982579103.issue41389@roundup.psfhosted.org>
In-reply-to
Content
Well, this isn't a help desk ;-) You may want instead to detail your problem on, say, StackOverflow, or the general Python mailing list.

Please note that I don't know what your "problem" _is_:  you haven't said. You posted some numbers that didn't make sense to you, and made unwarranted extrapolations from those (for example, no, those numbers won't get worse if you let the program run a million times longer).

So you should spell out what the "real" problem is. This shows signs of being an "XY problem":

    http://xyproblem.info/
    
For example, now you say:

> Calling gc.collect() on regular intervals doesn't seem
> to work consistently

That's news to me. The code you posted shows quite different behavior when FORCE_GC is set.

But if it's true that calling gc.collect() regularly doesn't alleviate "the real problem" (whatever that may be!), then that shows the opposite of what you appear to be assuming: that Python's cyclic gc is the root of the cause. collect() _will_ reclaim every scrap of RAM that's actually trash at the time it's called. So if calling that doesn't help, the problem is almost certainly NOT that trash isn't getting reclaimed. Something else is the cause.

Examples: it's not actually trash. It is, and gc collects it, but is unable to return it to the C library from which the memory came. It is returned to the C library, but that in turn is unable to return the memory to the OS. It is returned to the OS, but the OS decides to leave its virtual address space mapped to the process for now.

Details not only matter, they _can_ be everything when dealing with the multiple layers of memory management on modern machines. Waiting for a "general insight" is probably futile here :-(
History
Date User Action Args
2020-07-27 23:59:15tim.peterssetrecipients: + tim.peters, ian_osh
2020-07-27 23:59:15tim.peterssetmessageid: <1595894355.79.0.304982579103.issue41389@roundup.psfhosted.org>
2020-07-27 23:59:15tim.peterslinkissue41389 messages
2020-07-27 23:59:15tim.peterscreate