New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sys.exit() in a multiprocessing.Process does not align with Python behavior #79908
Comments
When a function is executed by a multiprocessing.Process and uses sys.exit, from functools import partial
from multiprocessing import get_context
import sys
def run(ctx, fn):
p = ctx.Process(target=fn)
p.start()
p.join()
return p.exitcode
if __name__ == '__main__':
ctx = get_context('fork')
print(run(ctx, partial(sys.exit, 2)))
print(run(ctx, partial(sys.exit, None)))
print(run(ctx, sys.exit))
ctx = get_context('spawn')
print(run(ctx, partial(sys.exit, 2)))
print(run(ctx, partial(sys.exit, None)))
print(run(ctx, sys.exit))
ctx = get_context('forkserver')
print(run(ctx, partial(sys.exit, 2)))
print(run(ctx, partial(sys.exit, None)))
print(run(ctx, sys.exit)) when executed results in $ python exit.py
2
1
1
2
1
1
2
1
1 but when Python itself is executed we see different behavior $ for arg in 2 None ''; do python -c "import sys; sys.exit($arg)"; echo $?; done
2
0
0 The documentation states
The relevant line in multiprocessing (https://github.com/python/cpython/blame/1cffd0eed313011c0c2bb071c8affeb4a7ed05c7/Lib/multiprocessing/process.py#L307) |
The same behavior on 3.8 and 3.5 |
I believe the mentality behind multiprocessing.Process triggering an exit code of 1 when sys.exit() is invoked inside its process is to indicate a non-standard exit out of its execution. There may yet be other side effects that could be triggered by having a sys.exit(0) translate into an exit code of 0 from the Process's process -- and we might not notice them with the current tests. Was there a particular use case that motivates this suggested change? |
Can I ask what this is based on? I did a pretty good amount of digging but didn't find any justification for it. It just seems like a simple oversight to me.
This is definitely a behavior change and will break any code that currently relies on
I have a wrapper library that invokes arbitrary user code and attempts to behave as if that code was executed in a vanilla Python process, to include propagating the correct exit code. Currently I have a workaround here: https://github.com/chrahunt/quicken/blob/2dd00a5f024d7b114b211aad8a2618ec8f101956/quicken/_internal/server.py#L344-L353, but it would be nice to get rid of it in 5-6 years if this fix gets in and the non-conformant Python versions fall out of support. :) |
Any other concerns here? |
Sorry for the delay. I've now merged the PR for 3.9. Since this is a slight behaviour change, and the original issue was easy to workaround, I won't backport it. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: