classification
Title: Broken Pipe in Server of Manager in multiprocessing when finalizing, sometimes
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, mutax
Priority: normal Keywords: patch

Created on 2021-07-12 22:07 by mutax, last changed 2021-07-13 13:25 by Mark.Shannon.

Files
File name Uploaded Description Edit
race2.py mutax, 2021-07-12 22:07
Messages (1)
msg397358 - (view) Author: Florian Streibelt (mutax) Date: 2021-07-12 22:07
It seems that the shutdown() method of the Server in in managers.py is already sending the result of its execution directly into the connection, and then the handle_request() method is trying to send ('#RETURN', None) yet again after returning from the call to shutdown() which sometimes yields a Broken Pipe Exception.

with the following line in handle_request():
    result = func(c, *args, **kwds)
a call to shutown() is made, and in shutdown() I see:
    c.send(('#RETURN', None))

and back in handle_request():
    msg = ('#RETURN', result)
…
    c.send(msg)

attached you find a minimal example that shows this behaviour sometimes, if logging is set to not more than DEBUG. I am guessing that the Broken Pipe depends on a race with the other side being closed.

Although the race condition does not look like having an impact, it is highly confusing when debugging. 


log:
      INFO           MainProcess         util.py:54  sending shutdown message to manager
     DEBUG        BaseManager-12         util.py:50  manager received shutdown message
      INFO        BaseManager-12         util.py:54  Failure to send message: ('#RETURN', None)
      INFO        BaseManager-12         util.py:54  process shutting down
      INFO        BaseManager-12         util.py:54   ... request was (None, 'shutdown', (), {})
     DEBUG        BaseManager-12         util.py:50  running all "atexit" finalizers with priority >= 0
      INFO        BaseManager-12         util.py:54   ... exception was BrokenPipeError(32, 'Broken pipe')
     DEBUG        BaseManager-12         util.py:50  running the remaining "atexit" finalizers
      INFO        BaseManager-12         util.py:54  process exiting with exitcode 0
History
Date User Action Args
2021-07-13 13:25:37Mark.Shannonsetpull_requests: - pull_request25654
2021-07-13 13:24:03Mark.Shannonsetkeywords: + patch
nosy: + Mark.Shannon

pull_requests: + pull_request25654
stage: patch review
2021-07-12 22:07:45mutaxcreate