Title: Incompatibilities with the socketserver and multiprocessing packages
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: davin, pitrou, rbprogrammer
Priority: normal Keywords: patch

Created on 2018-01-21 12:55 by rbprogrammer, last changed 2018-05-06 21:24 by rbprogrammer.

File name Uploaded Description Edit rbprogrammer, 2018-01-21 12:55 Script that exercises the bug, and demonstrates the proposed fix.
Pull Requests
URL Status Linked Edit
PR 5258 open rbprogrammer, 2018-01-21 13:01
Messages (7)
msg310371 - (view) Author: Michael Durso (rbprogrammer) * Date: 2018-01-21 12:55
This is more of an issue with socketserver rather than multiprocessing.  I could not find socketserver experts to add to the nosy list, but I've included the multiprocessing experts for completeness.

Under specific conditions, multiprocessing queues cease to work when modules like socketserver is used, more specifically when fork() is used.  This makes sense that things can break when fork() is used since a simple forked process has things like file descriptors and threads invalidated.  The socketserver.ForkingMixIn class, which has very reasonable use-cases, causes multiprocessing queues and threads to break.  A script that replicates and simulates the issue is attached.

The attached script sets up a multiprocessing queue used for logging across all sub-processes, then uses a socketserver.ForkingTCPServer to handle new client connection.  What's happening is the fork() is invalidating the thread in the QueueHandler in each new sub-process (for each new client conncetion).  It turns the QueueHandler into a dead object.

In this scenario, the only indication the problem exists is the fact that log messages are not actually logged.  But I think it's plausible that more than just log data could be lost.

When a non-modified version of CPython, the script should run and produce a log file.  However, logs from the client connection handler (MyClientHandler) will be lost and never sent to the logging handler that performs the actual file writing.  To exercise the proposed fix to CPython, build the changes to and then change the script's "USE_FIX" flag to "True" and run the script.
msg313652 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-12 14:18
Hi Michael, sorry for the silence on this issue.  I think the ProcessingMixIn feature is a good idea, I'll give the PR a review when I get the time.
msg313876 - (view) Author: Michael Durso (rbprogrammer) * Date: 2018-03-15 11:45
Hi Antoine, no worries for the delay.

I've been looking into why the AppVeyor check is failing.  I believe it to be an issue with the test harness in that it changes the environment during the tests, but only in the Windows case.  I have been thinking about ignoring the failing tests in Windows and create new tests just for the Windows case.  But I'm not sure that's the best approach.  Ideally the test harness should be able to either not modify the environment of the process or revert the changes it has made.

I'm not a strong Windows developer, so with my crude Windows development environment I'm struggling to see why the tests work in my Linux dev environment but not my Windows VM one.
msg315851 - (view) Author: Michael Durso (rbprogrammer) * Date: 2018-04-27 17:56
Hi Antoine, were you able to check out the pull request?  I'd like to get the issues resolved so the code can be merged into the next version.

msg315914 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-29 19:16
Hi Michael, sorry, I haven't had a chance yet. I'll try to make some time soon, I hope you don't mind the delay :-S
msg316021 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-05-01 21:25
Posted a review now.
msg316244 - (view) Author: Michael Durso (rbprogrammer) * Date: 2018-05-06 21:24
I added more commits to the PR based on your comments.  Although I cannot figure out why on the new Process objects create Threads that do not get cleaned up.
Date User Action Args
2018-05-06 21:24:08rbprogrammersetmessages: + msg316244
2018-05-01 21:25:02pitrousetmessages: + msg316021
2018-04-29 19:16:09pitrousetmessages: + msg315914
2018-04-27 17:56:46rbprogrammersetmessages: + msg315851
2018-03-15 11:45:58rbprogrammersetmessages: + msg313876
2018-03-12 14:18:32pitrousetmessages: + msg313652
versions: - Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7
2018-01-21 13:01:56rbprogrammersetkeywords: + patch
stage: patch review
pull_requests: + pull_request5104
2018-01-21 12:55:27rbprogrammercreate