Author inada.naoki
Recipients barry, benjamin.peterson, brainfvck, davin, inada.naoki, lukasz.langa, nascheme, pitrou, rhettinger, serhiy.storchaka, tim.peters, vstinner, yselivanov
Date 2017-10-12.04:27:13
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1507782433.53.0.213398074469.issue31558@psf.upfronthosting.co.za>
In-reply-to
Content
> So my understand is that gc frees some objects and makes some memory pages becomes available to allocate in child process. Allocation on the shared memory pages will cause the copy-on-write even without gc.
>
> Though this behavior may have better name?

OK, now I got what you're talking.
I don't know proper name about it.  I call it as "memory hole" for now.

But I don't think "memory hole" is big problem, because we already has refcount.
Say there are 100 function objects in one page, and 99 of them are never used. But when 1 of them are called, the page is unshared.

Solving memory hole issue is easy: just stop allocating new object from existing pages.
But I don't think it's worth enough because of refcount issue.

Instead of trying "share most data", I recommend to "use small number of processes" approach.

In my company, we don't use "prefork", but "--lazy-app" option of uWSGI for graceful reloading. (e.g. "afterfork")
But since we use nginx in front of uWSGI, # of uWSGI worker is just 2* CPU cores.  We can serve to massive clients from only 16~32 processes.

So I prefer optimizing normal memory usage.  It is good for all applications, not only "prefork" applications.

In this case, I'm +1 to gc.freeze() proposal because it can be used for single process applications.
History
Date User Action Args
2017-10-12 04:27:13inada.naokisetrecipients: + inada.naoki, tim.peters, barry, nascheme, rhettinger, pitrou, vstinner, benjamin.peterson, lukasz.langa, serhiy.storchaka, yselivanov, davin, brainfvck
2017-10-12 04:27:13inada.naokisetmessageid: <1507782433.53.0.213398074469.issue31558@psf.upfronthosting.co.za>
2017-10-12 04:27:13inada.naokilinkissue31558 messages
2017-10-12 04:27:13inada.naokicreate