This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Multiprocessing.Server.serve_forever runs sys.exit()
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bar.harel, davin, pitrou
Priority: normal Keywords:

Created on 2021-12-09 09:19 by bar.harel, last changed 2022-04-11 14:59 by admin.

Messages (2)
msg408090 - (view) Author: Bar Harel (bar.harel) * Date: 2021-12-09 09:19
The Python docs recommends using `.serve_forever()` for running a multiprocessing manager.

The problem is, that unlike any equivalent `.serve_forever()`, this specific one calls `sys.exit()` and exits the program. This in turn may cause issues if that's not the only use of the Python program, as seen here:
https://stackoverflow.com/questions/70284538/overriding-a-built-in-class-method-that-another-class-i-want-to-inherit-from-rel

3 possible solutions:

1. Remove `sys.exit()`: This will cause `shutdown()` to not actually shut down. The function counts on the `.accepter()` daemon thread to close, but if other code is preventing that process from closing, it will continue uninterrupted.

2. Remove `sys.exit()` and check the `stop_event` inside the `.accepter()` daemon thread: This will only partially shut down. The socket will still listen and will only close when the listener receives a command. An option to solve it would be to dispatch a shutdown command, or close the Listener forcibly using `.close()`. All in all, I believe this is the best option.

3. Discourage the use of `serve_forever()`: Right now the way it looks, server is not implemented too well. Its `serve_forever()` function is endorsed by the docs, but the class itself is completely undocumented, with no documentation on any of its functions, and it shuts down the process calling the function when it finishes which is dubious at best. There are plenty more issues scattered around the bug tracker, one of them about its finalization due to the use of daemon threads. Everything can be implemented using start() together with an event. The only reason I don't like this option, is because `serve_forever()` does seem convenient and can be fixed per option 2.

I'll start implementing option 2 in the background, together with adding documentation, but I would enjoy gathering more feedback.
msg408092 - (view) Author: Bar Harel (bar.harel) * Date: 2021-12-09 09:33
Regarding backwards compatibility, the only issue that can arise from solutions 1 or 2, is if someone uses a bare `except` statement for finalization around `serve_forever` instead of using `finally`.

Keep in mind the function is not documented and there is no good solution for the breakage as you're generally not supposed to catch SystemExit either way. We can make a small search for open source usage of `serve_forever` and see if catching SystemExit is a common idiom. Even if it's common, I still believe this breakage is acceptable.
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90180
2021-12-09 09:56:20bar.harelsettype: behavior
components: + Library (Lib)
2021-12-09 09:33:37bar.harelsetmessages: + msg408092
2021-12-09 09:19:49bar.harelcreate