Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_concurrent_futures failures on Windows: IOError('[Errno 232] The pipe is being closed') on _send_bytes() #57272

Closed
vstinner opened this issue Sep 29, 2011 · 9 comments
Closed

test_concurrent_futures failures on Windows: IOError('[Errno 232] The pipe is being closed') on _send_bytes() #57272

vstinner opened this issue Sep 29, 2011 · 9 comments
Labels
stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

BPO 13063
Nosy @amauryfa, @pitrou, @vstinner
Files
  • error_no_data.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2011-10-12.19:23:40.705>
created_at = <Date 2011-09-29.19:07:13.699>
labels = ['tests', 'type-bug', 'library']
title = "test_concurrent_futures failures on Windows: IOError('[Errno 232] The pipe is being closed') on _send_bytes()"
updated_at = <Date 2011-10-12.19:23:40.704>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2011-10-12.19:23:40.704>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2011-10-12.19:23:40.705>
closer = 'vstinner'
components = ['Library (Lib)', 'Tests']
creation = <Date 2011-09-29.19:07:13.699>
creator = 'vstinner'
dependencies = []
files = ['23309']
hgrepos = []
issue_num = 13063
keywords = ['patch']
message_count = 9.0
messages = ['144612', '144613', '144874', '144887', '145077', '145078', '145125', '145137', '145422']
nosy_count = 5.0
nosy_names = ['amaury.forgeotdarc', 'pitrou', 'vstinner', 'neologix', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13063'
versions = ['Python 3.2', 'Python 3.3']
    @vstinner
    Copy link
    Member Author 

    Re-running test 'test_concurrent_futures' in verbose mode
test_killed_child (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 5.79s ok
test_map (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 3.63s ok
test_map_exception (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 4.05s ok
test_map_timeout (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 9.64s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 3.60s ok
test_submit (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 3.63s ok
test_submit_keyword (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 4.09s ok
test_map (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.13s ok
test_map_exception (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.12s ok
test_map_submits_without_iteration (test.test_concurrent_futures.ThreadPoolExecutorTest)
Tests verifying issue 11777. ... 0.14s ok
test_map_timeout (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 6.11s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_submit (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_submit_keyword (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.11s ok
test_all_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 3.74s ok
test_first_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 5.18s ok
test_first_completed_some_already_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 5.02s ok
test_first_exception (test.test_concurrent_futures.ProcessPoolWaitTests) ... 6.57s ok
test_first_exception_one_already_failed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 6.37s ok
test_first_exception_some_already_complete (test.test_concurrent_futures.ProcessPoolWaitTests) ... 5.15s ok
test_timeout (test.test_concurrent_futures.ProcessPoolWaitTests) ... 10.94s ok
test_all_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 0.12s ok
test_first_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.62s ok
test_first_completed_some_already_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.61s ok
test_first_exception (test.test_concurrent_futures.ThreadPoolWaitTests) ... 3.12s ok
test_first_exception_one_already_failed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 2.10s ok
test_first_exception_some_already_complete (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.60s ok
test_timeout (test.test_concurrent_futures.ThreadPoolWaitTests) ... 6.12s ok
test_no_timeout (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 3.62s ok
test_zero_timeout (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 5.65s ok
test_no_timeout (test.test_concurrent_futures.ThreadPoolAsCompletedTests) ... 0.12s ok
test_zero_timeout (test.test_concurrent_futures.ThreadPoolAsCompletedTests) ... 2.11s ok
test_cancel (test.test_concurrent_futures.FutureTests) ... ok
test_cancelled (test.test_concurrent_futures.FutureTests) ... ok
test_done (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_already_cancelled (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_already_failed (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_already_successful (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_raises (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_with_cancel (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_with_exception (test.test_concurrent_futures.FutureTests) ... ok
test_done_callback_with_result (test.test_concurrent_futures.FutureTests) ... ok
test_exception_with_success (test.test_concurrent_futures.FutureTests) ... ok
test_exception_with_timeout (test.test_concurrent_futures.FutureTests) ... ok
test_repr (test.test_concurrent_futures.FutureTests) ... ok
test_result_with_cancel (test.test_concurrent_futures.FutureTests) ... ok
test_result_with_success (test.test_concurrent_futures.FutureTests) ... ok
test_result_with_timeout (test.test_concurrent_futures.FutureTests) ... ok
test_running (test.test_concurrent_futures.FutureTests) ... ok
test_context_manager_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 3.58s ok
test_del_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 3.92s ok
test_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 5.60s FAIL
test_processes_terminate (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 3.67s ok
test_run_after_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 0.01s ok
test_context_manager_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.02s ok
test_del_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.03s ok
test_interpreter_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 1.68s ok
test_run_after_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.00s ok
test_threads_terminate (test.test_concurrent_futures.ThreadPoolShutdownTest) ... Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\queues.py", line 267, in _feed
    send(obj)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 231, in send
    self._send_bytes(memoryview(buf))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 299, in _send_bytes
    overlapped = win32.WriteFile(self._handle, buf, overlapped=True)
IOError: [Errno 232] The pipe is being closed
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\queues.py", line 267, in _feed
    send(obj)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 231, in send
    self._send_bytes(memoryview(buf))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 299, in _send_bytes
    overlapped = win32.WriteFile(self._handle, buf, overlapped=True)
IOError: [Errno 232] The pipe is being closed
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\queues.py", line 267, in _feed
    send(obj)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 231, in send
    self._send_bytes(memoryview(buf))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 299, in _send_bytes
    overlapped = win32.WriteFile(self._handle, buf, overlapped=True)
IOError: [Errno 232] The pipe is being closed
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\queues.py", line 267, in _feed
    send(obj)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 231, in send
    self._send_bytes(memoryview(buf))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\connection.py", line 299, in _send_bytes
    overlapped = win32.WriteFile(self._handle, buf, overlapped=True)
IOError: [Errno 232] The pipe is being closed
Warning -- threading._dangling was modified by test_concurrent_futures
test test_concurrent_futures failed
0.01s ok

    ======================================================================
    FAIL: test_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_concurrent_futures.py", line 109, in test_interpreter_shutdown
    self.assertFalse(err)
AssertionError: b'Traceback (most recent call last):\n  File "D:\\cygwin\\home\\db3l\\buildarea\\3.x.bolen-windows\\build\\lib\\multiprocessing\\queues.py", line 267, in _feed\n    send(obj)\n  File "D:\\cygwin\\home\\db3l\\buildarea\\3.x.bolen-windows\\build\\lib\\multiprocessing\\connection.py", line 231, in send' is not false

    Ran 59 tests in 133.632s

    http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/5243/steps/test/logs/stdio

    @vstinner vstinner added the stdlib Python modules in the Lib dir label Sep 29, 2011
    @vstinner
    Copy link
    Member Author

    See also issue bpo-13059.

    @pitrou
    Copy link
    Member

    pitrou commented Oct 4, 2011

    I think the solution would be to map ERROR_NO_DATA (232 - "The pipe is being closed") to EPIPE. Attached patch.

    @pitrou pitrou added tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error labels Oct 4, 2011
    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 4, 2011

    Attached patch.

    Could you please explain your change in generrmap.c in a comment? For example, just add a reference to this issue.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 7, 2011

    New changeset d8d8374ddbcc by Antoine Pitrou in branch '3.2':
    Issue bpo-13063: the Windows error ERROR_NO_DATA (numbered 232 and described
    http://hg.python.org/cpython/rev/d8d8374ddbcc

    New changeset 3784b6000640 by Antoine Pitrou in branch 'default':
    Issue bpo-13063: the Windows error ERROR_NO_DATA (numbered 232 and described
    http://hg.python.org/cpython/rev/3784b6000640

    @pitrou
    Copy link
    Member

    pitrou commented Oct 7, 2011

    Hopefully this will fix the random failures (EPIPE is caught somewhere down in multiprocessing.Queue._feed).

    @pitrou
    Copy link
    Member

    pitrou commented Oct 7, 2011

    Mmh, for some reason it seems it was not enough:
    http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/5328/steps/test/logs/stdio

    @pitrou
    Copy link
    Member

    pitrou commented Oct 7, 2011

    Ah, this is because IOError is raised because of WindowsError, thus there's no errno translation.
    Ok, I'd like to wait for PEP-3151 to be merged, then, because it should solve this.

    @vstinner
    Copy link
    Member Author

    The issue has been fixed by the merge of the PEP-3151: I now get a BrokenPipeError with errno=32 and winerror=232.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @vstinner @pitrou