Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_subprocess: sporadic failure of test_communicate_epipe() on Windows 8 #63811

Closed
vstinner opened this issue Nov 15, 2013 · 11 comments
Closed
Labels
OS-windows tests Tests in the Lib/test dir

Comments

@vstinner
Copy link
Member

BPO 19612
Nosy @gvanrossum, @pitrou, @vstinner, @tjguk
Files
  • communicate_epipe.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2014-07-28.22:10:58.454>
    created_at = <Date 2013-11-15.16:29:59.634>
    labels = ['tests', 'OS-windows']
    title = 'test_subprocess: sporadic failure of test_communicate_epipe() on Windows 8'
    updated_at = <Date 2014-07-28.22:10:58.453>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2014-07-28.22:10:58.453>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2014-07-28.22:10:58.454>
    closer = 'vstinner'
    components = ['Tests', 'Windows']
    creation = <Date 2013-11-15.16:29:59.634>
    creator = 'vstinner'
    dependencies = []
    files = ['34131']
    hgrepos = []
    issue_num = 19612
    keywords = ['patch', 'buildbot']
    message_count = 11.0
    messages = ['202959', '204259', '211501', '211519', '211557', '211558', '211563', '211565', '212748', '224196', '224198']
    nosy_count = 6.0
    nosy_names = ['gvanrossum', 'pitrou', 'vstinner', 'tim.golden', 'neologix', 'python-dev']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue19612'
    versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

    @vstinner
    Copy link
    Member Author

    The test failed on a buildbot, see the message below. By the way, the test should use a value based on test.support.PIPE_MAX_SIZE rather than an hardcoded size of 2**20 bytes.

    http://buildbot.python.org/all/builders/x86%20Windows%20Server%202008%20%5BSB%5D%203.x/builds/1661/steps/test/logs/stdio

    test_communicate_epipe (test.test_subprocess.ProcessTestCase) ... ERROR
    test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCase) ... ok

    ======================================================================
    ERROR: test_communicate_epipe (test.test_subprocess.ProcessTestCase)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "E:\home\cpython\buildslave\x86\3.x.snakebite-win2k8r2sp1-x86\build\lib\test\test_subprocess.py", line 1095, in test_communicate_epipe
        p.communicate(b"x" * 2**20)
      File "E:\home\cpython\buildslave\x86\3.x.snakebite-win2k8r2sp1-x86\build\lib\subprocess.py", line 952, in communicate
        stdout, stderr = self._communicate(input, endtime, timeout)
      File "E:\home\cpython\buildslave\x86\3.x.snakebite-win2k8r2sp1-x86\build\lib\subprocess.py", line 1188, in _communicate
        self.stdin.write(input)
    OSError: [Errno 22] Invalid argument

    (...)
    test_communicate_epipe (test.test_subprocess.ProcessTestCase) ... ok
    test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCase) ... ok

    @vstinner vstinner added tests Tests in the Lib/test dir OS-windows labels Nov 15, 2013
    @vstinner vstinner changed the title test_subprocess: sporadic failure of test_communicate_epipe() test_subprocess: sporadic failure of test_communicate_epipe() on Windows 8 Nov 24, 2013
    @gvanrossum
    Copy link
    Member

    I don't know much about subprocess.py and I don't have access to Windows 8. But it looks like the kind of thing that might happen if the other end of the "pipe" is closed, which might happen if the subprocess exits early, either because it's just fast or because it's encountered some other bug.

    @vstinner
    Copy link
    Member Author

    If I add a sleep of 1 second before stdin.write(), the test always fail with OSError(22, "Invalid parameter"). So it becomes obvious that the write fails if the process already exited.

    Attached patch should fix the issue. Can someone review it?

    The test fails also on Python 3.3.

    @gvanrossum
    Copy link
    Member

    LGTM. Please commit in both repos and add to the list of cherrypicks for
    Larry.

    On Tue, Feb 18, 2014 at 1:40 AM, STINNER Victor <report@bugs.python.org>wrote:

    STINNER Victor added the comment:

    If I add a sleep of 1 second before stdin.write(), the test always fail
    with OSError(22, "Invalid parameter"). So it becomes obvious that the write
    fails if the process already exited.

    Attached patch should fix the issue. Can someone review it?

    The test fails also on Python 3.3.

    ----------
    keywords: +patch
    versions: +Python 3.3
    Added file: http://bugs.python.org/file34131/communicate_epipe.patch


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue19612\>


    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Feb 18, 2014

    New changeset 83013a7be836 by Victor Stinner in branch '3.3':
    Issue bpo-19612: On Windows, subprocess.Popen.communicate() now ignores
    http://hg.python.org/cpython/rev/83013a7be836

    New changeset 423cb79640eb by Victor Stinner in branch 'default':
    (Merge 3.3) Issue bpo-19612: On Windows, subprocess.Popen.communicate() now
    http://hg.python.org/cpython/rev/423cb79640eb

    @vstinner
    Copy link
    Member Author

    LGTM.

    Thanks for the review.

    Please commit in both repos and add to the list of cherrypicks for
    Larry.

    Both repos? You mean Tulip? This issue is specific to subprocess not asyncio (asyncio.subprocess).

    This sporadic bug is not new. IMO it exists since Python 3.3.

    I'm not sure that it's need to hurry to fix it in Python 3.4.0.

    @gvanrossum
    Copy link
    Member

    Sorry, forget everything I said. (That includes the LGTM, sadly -- as I wrote earlier I don't know this code very well.)

    @vstinner
    Copy link
    Member Author

    Sorry, forget everything I said. (That includes the LGTM, sadly -- as I wrote earlier I don't know this code very well.)

    Don't worry, I'm confident in my change :)

    @vstinner
    Copy link
    Member Author

    vstinner commented Mar 4, 2014

    Oh, it looks like my patch is not enough: the test still fails sometimes :-/ It looks like the process was still running (poll() didn't return None) whereas stdin.write() got an OSError(errno.EINVAL).

    http://buildbot.python.org/all/builders/x86%20Windows%20Server%202008%20%5BSB%5D%203.x/builds/2408/steps/test/logs/stdio

    ======================================================================
    ERROR: test_communicate_epipe (test.test_subprocess.ProcessTestCase)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "E:\home\cpython\buildslave\x86\3.x.snakebite-win2k8r2sp1-x86\build\lib\test\test_subprocess.py", line 1095, in test_communicate_epipe
        p.communicate(b"x" * 2**20)
      File "E:\home\cpython\buildslave\x86\3.x.snakebite-win2k8r2sp1-x86\build\lib\subprocess.py", line 951, in communicate
        stdout, stderr = self._communicate(input, endtime, timeout)
      File "E:\home\cpython\buildslave\x86\3.x.snakebite-win2k8r2sp1-x86\build\lib\subprocess.py", line 1187, in _communicate
        self.stdin.write(input)
    OSError: [Errno 22] Invalid argument

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 28, 2014

    New changeset 039ac3f01c4e by Victor Stinner in branch '2.7':
    Issue bpo-19612: subprocess.communicate() now also ignores EINVAL when using at
    http://hg.python.org/cpython/rev/039ac3f01c4e

    @vstinner
    Copy link
    Member Author

    I didnd't see the failure recently on Windows, so I close the issue.

    I backported the fix on Python 2.7 for the issue bpo-21946.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    OS-windows tests Tests in the Lib/test dir
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants