classification
Title: muliprocessing shutdown infinite loop
Type: behavior Stage:
Components: Versions: Python 2.6
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: jnoller Nosy List: asksol, davin, drraid, ezio.melotti, jnoller
Priority: normal Keywords:

Created on 2010-03-15 01:51 by drraid, last changed 2015-01-29 22:53 by davin. This issue is now closed.

Messages (5)
msg101078 - (view) Author: brandon (drraid) Date: 2010-03-15 01:51
Multiprocessing goes into an infinite loop during shutdown, trying to connect to a remote queue - I *think* during finalization. 

The actual loop appears to be the while(1) in connection.py line 251, and I think it is being called initially from manager.py finalization.

I can reliably reproduce but my code base is large and ugly, and I am still trimming down to a nice clean sample to submit.
msg101081 - (view) Author: brandon (drraid) Date: 2010-03-15 02:01
After getting into the infinite loop, here's exception from CTRL-C (posted just to show stack trace -- i am still working on getting come sample code together -- meant to show it is infact in that while(1) a 251 in connection.py):

KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/util.py", line 260, in _exit_function
    _run_finalizers(0)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/util.py", line 235, in _run_finalizers
    finalizer()
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/util.py", line 174, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/managers.py", line 773, in _decref
    conn = _Client(token.address, authkey=authkey)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/connection.py", line 134, in Client
    c = SocketClient(address)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/connection.py", line 258, in SocketClient
    time.sleep(0.01)
KeyboardInterrupt
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/util.py", line 260, in _exit_function
    _run_finalizers(0)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/util.py", line 235, in _run_finalizers
    finalizer()
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/util.py", line 174, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/managers.py", line 773, in _decref
    conn = _Client(token.address, authkey=authkey)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/connection.py", line 134, in Client
    c = SocketClient(address)
  File "/home/drraid/code/fuzzer/fuzznet/multiprocessing/connection.py", line 258, in SocketClient
    time.sleep(0.01)
KeyboardInterrupt
msg118017 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2010-10-05 15:13
Did you finish the code to reproduce the problem?
msg180731 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-01-27 01:35
Can you still reproduce the issue?
msg234998 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015-01-29 22:53
At nearly 5 years of age, this issue unfortunately never saw a viable demonstration of how to provoke the reported behavior.  Hopefully the reporter either (1) discovered the nature of the problem originated elsewhere and he was able to quickly fix it, or (2) won the lottery and merrily forgot all about this issue (and many other real-world issues).  Many things have changed in the multiprocessing module in the last 5 years and it is difficult to see how one might deliberately reproduce this now.  In the event that there is a reproducible issue like described herein, we hope that someone will step forward to open a new issue with more information.
History
Date User Action Args
2015-01-29 22:53:39davinsetstatus: open -> closed

nosy: + davin
messages: + msg234998

resolution: out of date
2013-01-27 01:35:24ezio.melottisetnosy: + ezio.melotti
messages: + msg180731
2010-10-05 15:13:22asksolsetnosy: + asksol
messages: + msg118017
2010-03-15 02:01:40drraidsetmessages: + msg101081
2010-03-15 01:52:26benjamin.petersonsetassignee: jnoller

nosy: + jnoller
2010-03-15 01:51:04drraidcreate