classification
Title: OSError in os.waitpid() on Windows [3.5.0 regression]
Type: behavior Stage:
Components: Library (Lib), Windows Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, paul.moore, python-dev, rocco.matano, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: 3.5regression

Created on 2015-09-15 05:52 by rocco.matano, last changed 2015-09-21 07:18 by vstinner. This issue is now closed.

Messages (6)
msg250723 - (view) Author: Rocco Matano (rocco.matano) Date: 2015-09-15 05:52
On windows and python up to 3.4 using the combination of os.spawnX(os.P_NOWAIT, ...) and os.waitpid() worked as expected, but with python 3.5 this no longer works. In fact os.waitpid() now raises an OSError when the child process terminates.
Running this simple script demonstrates that:

import sys
import os
import time

print(sys.version)

file = r'c:\windows\system32\cmd.exe'
args = [file, '/C', 'ping', '1.1.1.1', '-n', '1', '>NUL']
env = os.environ

t0 = time.time()
ret = os.spawnve(os.P_NOWAIT, file, args, env)
pid, status = os.waitpid(ret, 0)
t1 = time.time()
print("process took %.1f seconds" % (t1 - t0))

I get the following outputs:

C:\Users\rmatano\test_py35>py -3.4 test_waitpid.py
3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:25:23) [MSC v.1600 64 bit (AMD64)]
process took 3.6 seconds

C:\Users\rmatano\test_py35>py -3.5 test_waitpid.py
3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AMD64)]
Traceback (most recent call last):
  File "twp.py", line 13, in <module>
    pid, status = os.waitpid(ret, 0)
OSError: [Errno 0] Error

I guess this is a bug rather than a intended change in behavior, isn't it?
msg250728 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-09-15 07:24
This bug is due to issue 23285, which improved support for EINTR handling. os_waitpid_impl was changed to use the following do-while loop:


    do {
        Py_BEGIN_ALLOW_THREADS
        res = _cwait(&status, pid, options);
        Py_END_ALLOW_THREADS
    } while (res < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));
    if (res != 0)
        return (!async_err) ? posix_error() : NULL;

The last test should be (res < 0) instead of (res != 0). That's why you're getting a no-error exception.

It seems to me this entire loop should be removed. The Windows C runtime doesn't set errno to EINTR. In the case of _cwait it doesn't even use an alertable wait (i.e. it can't be interrupted by a regular asynchronous procedure call).
msg250735 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-09-15 08:26
New changeset 9a80c687c28d by Victor Stinner in branch '3.5':
Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on Windows.
https://hg.python.org/cpython/rev/9a80c687c28d
msg250738 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-09-15 08:34
Oops :-/ Yet another Python 3.5.0 regression, it's now fixed.

os.waitpid() was not tested at all on Windows. os.waitpid() is not the best option to wait for a subprocess completion. By the way, you should use the subprocess module which is more portable, is widely used, etc. IMHO os.spawn*() is more kept for backward compatibility.

> It seems to me this entire loop should be removed. The Windows C runtime doesn't set errno to EINTR. In the case of _cwait it doesn't even use an alertable wait (i.e. it can't be interrupted by a regular asynchronous procedure call).

Well, it looks like you are right: _cwait() cannot be interrupted on Windows. But I chose to only fix the if() after the loop... just in case.
msg250760 - (view) Author: Rocco Matano (rocco.matano) Date: 2015-09-15 12:18
I know that using os.spawn and and os.waitpid this way is not the best option, but a 3rd party tool i am using (scons) is doing it that way. So no scons with Python 3.5.0. (I am also aware that scons does not yet support Python 3.x officially.)
msg250762 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-09-15 12:26
> I know that using os.spawn and and os.waitpid this way is not the best option, but a 3rd party tool i am using (scons) is doing it that way.

No worry, it was just a comment. Anyway, the bug is now fixed and will be part of the next Python 3.5.1 release.
History
Date User Action Args
2015-09-21 07:18:47vstinnersetkeywords: + 3.5regression
2015-09-15 12:26:28vstinnersetmessages: + msg250762
2015-09-15 12:18:04rocco.matanosetmessages: + msg250760
2015-09-15 08:34:47vstinnersetstatus: open -> closed
title: OSError in os.waitpid -> OSError in os.waitpid() on Windows [3.5.0 regression]
resolution: fixed
versions: + Python 3.6
2015-09-15 08:34:16vstinnersetnosy: + vstinner
messages: + msg250738
2015-09-15 08:26:31python-devsetnosy: + python-dev
messages: + msg250735
2015-09-15 07:24:41eryksunsetnosy: + paul.moore, eryksun, tim.golden, zach.ware, steve.dower
messages: + msg250728
components: + Windows
2015-09-15 05:52:11rocco.matanocreate