Author inada.naoki
Recipients barry, benjamin.peterson, brainfvck, davin, inada.naoki, lukasz.langa, nascheme, pitrou, rhettinger, serhiy.storchaka, tim.peters, vstinner, yselivanov
Date 2017-10-12.00:46:13
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1507769175.92.0.213398074469.issue31558@psf.upfronthosting.co.za>
In-reply-to
Content
>> This is only useful if the parent process has a lot of memory that's never used by the child processes right? Otherwise, you would lose via refcounting COWs.
>
> What we saw in prod is that memory fragmentation caused by gc is the main reason of shared memory shrink.
>
> The memory fragmentation is figured out by doing a full collection before fork and keep it disabled, it'll make a bunch of copy-on-write in child process.

GC doesn't cause "memory fragmentation".
GC touches (writes) GC header and refcount.  It cause sharing memory shrink.
Maybe, you're wrongly understanding "memory fragmentation".

> This can't solve the copy-on-write caused by ref count, but we're thinking about freezing the ref count on those permanent objects too.

It may increase cost of refcount operation, because it makes all INCREF and DECREF bigger.
Note that this is only helps application using gc.freeze().  This shouldn't slow down all other applications.

> So this is useful if you did some warm-up work in parent process.

I don't understand this statement.

> Also it could speedup gc if you have large amount of permanent objects.

Yes, this helps not only "prefork" application, but also all long running applications
having large baseline data.
History
Date User Action Args
2017-10-12 00:46:16inada.naokisetrecipients: + inada.naoki, tim.peters, barry, nascheme, rhettinger, pitrou, vstinner, benjamin.peterson, lukasz.langa, serhiy.storchaka, yselivanov, davin, brainfvck
2017-10-12 00:46:15inada.naokisetmessageid: <1507769175.92.0.213398074469.issue31558@psf.upfronthosting.co.za>
2017-10-12 00:46:15inada.naokilinkissue31558 messages
2017-10-12 00:46:13inada.naokicreate