Author davin
Recipients davin, eryksun, neologix, pietvo, pitrou, sbt, tim.peters
Date 2016-07-14.01:54:44
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1468461286.48.0.680272415854.issue18966@psf.upfronthosting.co.za>
In-reply-to
Content
It is a general rule that when a process terminates, it likewise terminates all its threads (unless a thread has been explicitly detached).  How it goes about doing so is complicated.

Remember that POSIX threads have no concept of parent/child among themselves and all threads are viewed as a single pool.  The section "No parents, no children" at 
http://www.ibm.com/developerworks/library/l-posix1/ offers us motivation for why waiting on a pthread should be explicitly requested and not assumed as a default behavior.

There are numerous differences between what a UNIX-style process and a win32 process does at termination.  Though an older post from Microsoft, a strong sense of how complicated the process-termination-begets-thread-termination truly is can be had from reading https://blogs.msdn.microsoft.com/oldnewthing/20070503-00/?p=27003 which also helps reinforce the sentiment above (needs explicit instructions on what to do, no general solution can exist).  Whereas the prior provided some sense of motivation, this link walks us through ugly complications and consequences that arise.

The short of it is that the current use of os._exit() is most appropriate in multiprocessing.  Threads should be signaled that the process is terminating but we are not generally expected to wait on those threads.

These and many other reference-worthy links help support the call for atexit-like functionality to be exposed on multiprocessing.Process.  There have been multiple issues opened on the bug tracker ultimately requesting this enhancement (do a search for multiprocessing and atexit).  I think it's a very sensible enhancement (Antoine did too and perhaps still does) and worth taking the time to pursue.  As an aside, I wonder if an equivalent to pthread_cleanup_push should also be exposed on threading.Thread.

When it comes to documentation, I am of two minds.  There seem to be an increasing number of people coming to Python without much prior exposure to the concepts of threads and processes and so it would be wrong for us to ignore this reality.  On the flip side, attempting to convey all the core concepts of threads and processes and how they interact would result in a large documentation effort that ultimately few people would eagerly read to completion.  Adding a one-sentence caveat hiding somewhere in the docs won't do much to help.  Given this topic and a few other issues that have come up very recently, I suggest that a concise paragraph be added on the topic of using threads and processes together -- likely placed at the beginning of the docs on the Process class.  I think I'm up for taking a crack at that but I'd very much appreciate critical eyes to review it with me.





Per Eryk's point about the difference in multiprocessing's behavior when using spawn vs. fork, the explanation for why it's done that way is also described in the DeveloperWorks article I mentioned above.

Finally, per the original post from pietvo for future readers, not only is it *not* weird to start a Thread within a Process, it's a popular and useful technique.
History
Date User Action Args
2016-07-14 01:54:46davinsetrecipients: + davin, tim.peters, pitrou, pietvo, neologix, sbt, eryksun
2016-07-14 01:54:46davinsetmessageid: <1468461286.48.0.680272415854.issue18966@psf.upfronthosting.co.za>
2016-07-14 01:54:46davinlinkissue18966 messages
2016-07-14 01:54:44davincreate