Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSError in os.waitpid() on Windows [3.5.0 regression] #69305

Closed
RoccoMatano mannequin opened this issue Sep 15, 2015 · 6 comments
Closed

OSError in os.waitpid() on Windows [3.5.0 regression] #69305

RoccoMatano mannequin opened this issue Sep 15, 2015 · 6 comments
Labels
OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@RoccoMatano
Copy link
Mannequin

RoccoMatano mannequin commented Sep 15, 2015

BPO 25118
Nosy @pfmoore, @vstinner, @tjguk, @zware, @eryksun, @zooba, @RoccoMatano

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-09-15.08:34:47.048>
created_at = <Date 2015-09-15.05:52:11.104>
labels = ['type-bug', 'library', 'OS-windows']
title = 'OSError in os.waitpid() on Windows [3.5.0 regression]'
updated_at = <Date 2015-09-21.07:18:47.663>
user = 'https://github.com/RoccoMatano'

bugs.python.org fields:

activity = <Date 2015-09-21.07:18:47.663>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2015-09-15.08:34:47.048>
closer = 'vstinner'
components = ['Library (Lib)', 'Windows']
creation = <Date 2015-09-15.05:52:11.104>
creator = 'rocco.matano'
dependencies = []
files = []
hgrepos = []
issue_num = 25118
keywords = ['3.5regression']
message_count = 6.0
messages = ['250723', '250728', '250735', '250738', '250760', '250762']
nosy_count = 8.0
nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'python-dev', 'zach.ware', 'eryksun', 'steve.dower', 'rocco.matano']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25118'
versions = ['Python 3.5', 'Python 3.6']

@RoccoMatano
Copy link
Mannequin Author

RoccoMatano mannequin commented Sep 15, 2015

On windows and python up to 3.4 using the combination of os.spawnX(os.P_NOWAIT, ...) and os.waitpid() worked as expected, but with python 3.5 this no longer works. In fact os.waitpid() now raises an OSError when the child process terminates.
Running this simple script demonstrates that:

import sys
import os
import time

print(sys.version)

file = r'c:\windows\system32\cmd.exe'
args = [file, '/C', 'ping', '1.1.1.1', '-n', '1', '>NUL']
env = os.environ

t0 = time.time()
ret = os.spawnve(os.P_NOWAIT, file, args, env)
pid, status = os.waitpid(ret, 0)
t1 = time.time()
print("process took %.1f seconds" % (t1 - t0))

I get the following outputs:

C:\Users\rmatano\test_py35>py -3.4 test_waitpid.py
3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:25:23) [MSC v.1600 64 bit (AMD64)]
process took 3.6 seconds

C:\Users\rmatano\test_py35>py -3.5 test_waitpid.py
3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AMD64)]
Traceback (most recent call last):
  File "twp.py", line 13, in <module>
    pid, status = os.waitpid(ret, 0)
OSError: [Errno 0] Error

I guess this is a bug rather than a intended change in behavior, isn't it?

@RoccoMatano RoccoMatano mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 15, 2015
@eryksun
Copy link
Contributor

eryksun commented Sep 15, 2015

This bug is due to bpo-23285, which improved support for EINTR handling. os_waitpid_impl was changed to use the following do-while loop:

do {
    Py_BEGIN_ALLOW_THREADS
    res = _cwait(&status, pid, options);
    Py_END_ALLOW_THREADS
} while (res < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));
if (res != 0)
    return (!async_err) ? posix_error() : NULL;

The last test should be (res < 0) instead of (res != 0). That's why you're getting a no-error exception.

It seems to me this entire loop should be removed. The Windows C runtime doesn't set errno to EINTR. In the case of _cwait it doesn't even use an alertable wait (i.e. it can't be interrupted by a regular asynchronous procedure call).

@python-dev
Copy link
Mannequin

python-dev mannequin commented Sep 15, 2015

New changeset 9a80c687c28d by Victor Stinner in branch '3.5':
Issue bpo-25118: Fix a regression of Python 3.5.0 in os.waitpid() on Windows.
https://hg.python.org/cpython/rev/9a80c687c28d

@vstinner
Copy link
Member

Oops :-/ Yet another Python 3.5.0 regression, it's now fixed.

os.waitpid() was not tested at all on Windows. os.waitpid() is not the best option to wait for a subprocess completion. By the way, you should use the subprocess module which is more portable, is widely used, etc. IMHO os.spawn*() is more kept for backward compatibility.

It seems to me this entire loop should be removed. The Windows C runtime doesn't set errno to EINTR. In the case of _cwait it doesn't even use an alertable wait (i.e. it can't be interrupted by a regular asynchronous procedure call).

Well, it looks like you are right: _cwait() cannot be interrupted on Windows. But I chose to only fix the if() after the loop... just in case.

@vstinner vstinner changed the title OSError in os.waitpid OSError in os.waitpid() on Windows [3.5.0 regression] Sep 15, 2015
@RoccoMatano
Copy link
Mannequin Author

RoccoMatano mannequin commented Sep 15, 2015

I know that using os.spawn and and os.waitpid this way is not the best option, but a 3rd party tool i am using (scons) is doing it that way. So no scons with Python 3.5.0. (I am also aware that scons does not yet support Python 3.x officially.)

@vstinner
Copy link
Member

I know that using os.spawn and and os.waitpid this way is not the best option, but a 3rd party tool i am using (scons) is doing it that way.

No worry, it was just a comment. Anyway, the bug is now fixed and will be part of the next Python 3.5.1 release.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants