Author gregory.p.smith
Recipients John Jones, alex, benjamin.peterson, dhduvall, gennad, gregory.p.smith, neologix
Date 2017-03-18.02:34:27
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAGE7PN+BN8tBepPeN1diDU2XD6T5ieoDM17sSebRyR3kL-c90g@mail.gmail.com>
In-reply-to <1489776577.7.0.530513240047.issue20104@psf.upfronthosting.co.za>
Content
All I'm really saying is that someone who wants this should provide a
patch/PR with unittests. :)  I can help review and go from there.

It does make sense to me for it to be available as part of the subprocess
API if it is available at all, likely an alternative implementation of or
behavior flag to _posixsubprocess.fork_exec() with appropriate autoconf and
conditional compilation based on availability ifdefs.

On Fri, Mar 17, 2017 at 11:49 AM John Jones <report@bugs.python.org> wrote:

>
> John Jones added the comment:
>
> I agree with everything you're saying Gregory, however I don't think the
> significance of the memory doubling is as inconsequential as you might
> first think. For example, i have on my 64bit Linux system 128Gb of RAM, and
> a numpy table that's around 70Gb. Spawning a subprocess, even though memory
> is doubled for a very short period of time, is enough to raise a
> MemoryError, despite the subprocess i'm spawning using only 2 or 3Mb after
> the exec().
>
> I do appreciate that for most Python users however, they will not see much
> benefit from what I imagine is quite a lot of development work.
>
> FWIW, I did try the posix_spawn module, but i couldn't figure out how to
> write data to the stdin of a posix_spawn subprocess, and gave up in place
> of the commonly recommended solution to this problem (via StackExchange) of
> spawning lots of subprocesses before you put stuff in memory. Fortunately,
> for my problem, this was a possible solution. For others I think they're
> going to have to use posix_spawn, or an entirely different programming
> language if that doesn't work.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue20104>
> _______________________________________
>
History
Date User Action Args
2017-03-18 02:34:28gregory.p.smithsetrecipients: + gregory.p.smith, benjamin.peterson, alex, dhduvall, neologix, gennad, John Jones
2017-03-18 02:34:28gregory.p.smithlinkissue20104 messages
2017-03-18 02:34:27gregory.p.smithcreate