Author lesha
Recipients Connor.Wolf, Giovanni.Bajo, bobbyi, dan.oreilly, emptysquare, forest_atq, gregory.p.smith, ionelmc, jcea, lesha, neologix, nirai, nirs, pitrou, sbt, sdaoden, tshepang, vinay.sajip, vstinner
Date 2016-07-09.04:03:22
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1468037003.19.0.0105235949925.issue6721@psf.upfronthosting.co.za>
In-reply-to
Content
I work on a multi-million-line C++ codebase that uses fork() from multithreaded programs all over the place. We use `glog` ubiquitously.  

This bug here that spans 6 years and has affected dozens of people (conservatively) simply does not exist for us. That is because glog takes the pragmatic approach of sanitizing its mutex on fork:

https://github.com/google/glog/blob/4d391fe692ae6b9e0105f473945c415a3ce5a401/src/base/mutex.h#L249

In my opinion, "thou shalt never fork() in a threaded program" is impractical purism. It is good to be aware of the dangers that lie therein, but it is completely possible to safely spawn **subprocesses** from multithreaded programs on modern OSes like Linux.

Python's subprocess **ought** to be safe to use in threaded programs. Any issues with this (aside from `pre_exec_fn`, obviously) are bugs in `subprocess`. 

Here is a C++ implementation of the concept that can be safely used in threaded programs:

https://github.com/facebook/folly/blob/master/folly/Subprocess.cpp

Note that unlike Python's subprocess `pre_exec_fn`, the C++ analog is very loud in warning you about the scary world in which your closure will execute:

https://github.com/facebook/folly/blob/master/folly/Subprocess.h#L252

The point to my message is simple: there is a pragmatic way to save hundreds of debugging hours for users of Python. People are going to find it necessary to do such things from time to time, so instead of telling them to redesign their programs, it is better to give them a safer tool.

Taking the glog approach in `logging` has no cost to the standard library, but it does have real world benefits. 

Please don't block shipping such a fix.
History
Date User Action Args
2016-07-09 04:03:23leshasetrecipients: + lesha, gregory.p.smith, vinay.sajip, jcea, nirs, pitrou, vstinner, nirai, forest_atq, ionelmc, bobbyi, neologix, Giovanni.Bajo, sdaoden, tshepang, sbt, emptysquare, dan.oreilly, Connor.Wolf
2016-07-09 04:03:23leshasetmessageid: <1468037003.19.0.0105235949925.issue6721@psf.upfronthosting.co.za>
2016-07-09 04:03:23leshalinkissue6721 messages
2016-07-09 04:03:22leshacreate