classification
Title: process thread with implicit join is killed unexpectedly
Type: behavior Stage:
Components: Documentation Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Threads within multiprocessing Process terminate early
View: 18966
Assigned To: docs@python Nosy List: JA, docs@python, r.david.murray, tim.peters
Priority: normal Keywords:

Created on 2016-07-13 17:20 by JA, last changed 2016-07-13 19:58 by tim.peters. This issue is now closed.

Messages (5)
msg270329 - (view) Author: JA (JA) Date: 2016-07-13 17:20
On Ubuntu 16.04 
python '3.5.1+ (default, Mar 30 2016, 22:46:26) \n[GCC 5.3.1 20160330]'

The following code prints "hi" 4 times: 
import multiprocessing
import time
import threading    

class dumb(threading.Thread):
    def __init__(self):
        super(dumb, self).__init__()
    def run(self):
        while True:
            print("hi")
            time.sleep(1)    

def test():
    for i in range(2):
        bar = dumb()
        bar.start()
def main():
    p = []
    for i in range(2):
        p.append(multiprocessing.Process(target=test))
    for i in p:
        i.start()
    for i in p:
        i.join()
if __name__ == '__main__':
    main()

Note: The above code runs fine on Python 3.5.2 (64-bit) on Windows 10

Joining the threads in test fixes the problem: 
def test():
    p = []
    for i in range(2):
        bar = dumb()
        bar.start()
        p.append(bar)
    for i in p:
        i.join()
msg270330 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2016-07-13 17:24
Note:  this started on stackoverflow:

https://stackoverflow.com/questions/38356584/python-multiprocessing-threading-code-exits-early

I may be missing something obvious, but the only explanation I could think of for the behavior seen on Ubuntu is that the threads in the worker processes are being treated as daemon threads.  The program works as intended for me on Windows - but, of course, there's a universe of differences between spawning (Windows) and forking (Ubuntu) processes too.
msg270335 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2016-07-13 18:46
Curious:  under Python 2.7.11 on Windows, the threads also terminate early (they run "forever" - as intended - under 3.5.2).
msg270336 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-07-13 19:32
On my gentoo system it prints hi four times in 2.7, and 3.2 through 3.6.  I suspect multiprocessing is killing threads, daemon or not, when the main process thread ends.  I expect that's a feature, although I didn't find it documented.

Ah, found the explanation: issue 18966.  Antoine indicates this could indeed be considered a bug but is part of a larger issue.  If we aren't going to fix it, we should probably document it.
msg270337 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2016-07-13 19:58
Ah - good catch!  I'm closing this as a duplicate of bug18966.  The real mystery now is why the threads _don't_ terminate early under Windows 3.5.2 - heh.
History
Date User Action Args
2016-07-13 19:58:10tim.peterssetstatus: open -> closed
superseder: Threads within multiprocessing Process terminate early
resolution: duplicate
messages: + msg270337
2016-07-13 19:32:53r.david.murraysetversions: + Python 2.7, Python 3.6
nosy: + docs@python, r.david.murray

messages: + msg270336

assignee: docs@python
components: + Documentation, - Library (Lib)
2016-07-13 18:46:09tim.peterssetmessages: + msg270335
2016-07-13 17:27:53tim.peterssettype: behavior
components: + Library (Lib)
2016-07-13 17:24:23tim.peterssetnosy: + tim.peters
messages: + msg270330
2016-07-13 17:20:48JAcreate