test_io broken on PPC64 Linux #62035
Comments
|
Unoptimized debug build (configured using --with-pydebug). test_interrupted_write_retry_buffered (test.test_io.CSignalsTest) ... ERROR ====================================================================== Traceback (most recent call last):
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3219, in test_interrupted_write_retry_buffered
self.check_interrupted_write_retry(b"x", mode="wb")
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3203, in check_interrupted_write_retry
t.join()
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/threading.py", line 738, in join
raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started====================================================================== Traceback (most recent call last):
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3222, in test_interrupted_write_retry_text
self.check_interrupted_write_retry("x", mode="w", encoding="latin1")
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3203, in check_interrupted_write_retry
t.join()
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/threading.py", line 738, in join
raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started====================================================================== Traceback (most recent call last):
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3219, in test_interrupted_write_retry_buffered
self.check_interrupted_write_retry(b"x", mode="wb")
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3203, in check_interrupted_write_retry
t.join()
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/threading.py", line 738, in join
raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started====================================================================== Traceback (most recent call last):
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3222, in test_interrupted_write_retry_text
self.check_interrupted_write_retry("x", mode="w", encoding="latin1")
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/test_io.py", line 3203, in check_interrupted_write_retry
t.join()
File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/threading.py", line 738, in join
raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started |
|
What does the following say for you?
|
>>> import fcntl, os
>>> r, w = os.pipe()
>>> fcntl.fcntl(w, 1032)
1048576 |
|
Ah, right. That number is the pipe buffer size (1032 is F_GETPIPE_SZ). It's 65536 here, so when the test tries to write 1 million bytes on a pipe, the write blocks as expected (you can read the comments to understand why the test is doint that). But with a 1 MiB buffer size, the write doesn't block and therefore doesn't have to wait for the auxiliary thread to start and read from the pipe buffer. Something else, what does the following say:
|
>>> r, w = os.pipe()
>>> fcntl.fcntl(r, 1031, 1000)
65536 |
|
Ok, what does the following patch do for you: diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py
--- a/Lib/test/test_io.py
+++ b/Lib/test/test_io.py
@@ -3168,7 +3168,7 @@ class SignalsTest(unittest.TestCase):
select = support.import_module("select")
# A quantity that exceeds the buffer size of an anonymous pipe's
# write end.
- N = 1024 * 1024
+ N = 1024 * 1024 + 1
r, w = os.pipe()
fdopen_kwargs["closefd"] = False
# We need a separate thread to read from the pipe and allow the
@@ -3191,6 +3191,12 @@ class SignalsTest(unittest.TestCase):
signal.signal(signal.SIGALRM, alarm1)
try:
wio = self.io.open(w, **fdopen_kwargs)
+ if sys.platform.startswith('linux') and fcntl is not None:
+ # Issue python/cpython#62035: try to limit pipe size using F_SETPIPE_SZ
+ pipe_size = fcntl.fcntl(w, 1031, 4096)
+ if pipe_size >= N:
+ self.skipTest("can't set pipe size to less than %d bytes: %d"
+ % (N, pipe_size))
signal.alarm(1)
# Expected behaviour:
# - first raw write() is partial (because of the limited pipe buffer |
|
Why not simply increase the amount of data written instead of limiting By the way, there's support.PIPE_MAX_SIZE for that purpose. |
Hmm, indeed :-)
Hardwired to 3 MB. I wonder if it may broken soon. |
|
The patch limiting the pipe size resolves the test_io failure. Either of the approaches should work. |
On Linux, for non root users, it's limited to 1048576, and can be set I think 3MB should be more than enough, so I suggest to update the r, w = os.pipe() But F_GETPIPE_SZ is Linux only, and quite recent (since 2.6.35 apparently). |
|
Ok, here is a new patch updating PIPE_MAX_SIZE with a proper value if possible. |
|
The PIPE_MAX_SIZE patch also fixes the failure. |
The patch is correct, however I think it's a bit overkill, especially Anyway, this problem also affects all versions from 2.7, so |
Ok, so let's say 4MB + 1 then. |
|
New changeset 4b4ed1e11fd0 by Antoine Pitrou in branch '3.3': New changeset de35eae9048a by Antoine Pitrou in branch 'default': New changeset 09811ecd5df1 by Antoine Pitrou in branch '2.7': |
|
This should be fixed now. David, please re-open if the failure still occurs. |
|
I encountered similar issues to those discussed in this issue whilst compiling 3.4.1 on 'Red Hat Enterprise Linux Server release 6.5 (Santiago)' In particular, the following tests were failing - [root@lonlx90800 ~]# /local/0/python-3.4.1/bin/python3 /local/0/python-3.4.1/lib/python3.4/test/test_asyncio/test_subprocess.py Traceback (most recent call last):
File "/local/0/python-3.4.1/lib/python3.4/test/test_asyncio/test_subprocess.py", line 129, in test_broken_pipe
self.loop.run_until_complete(proc.communicate(large_data))
AssertionError: BrokenPipeError not raised====================================================================== Traceback (most recent call last):
File "/local/0/python-3.4.1/lib/python3.4/test/test_asyncio/test_subprocess.py", line 129, in test_broken_pipe
self.loop.run_until_complete(proc.communicate(large_data))
AssertionError: BrokenPipeError not raisedIn this case, the issues are being caused by the following kernel parameters that we have for our default build - #########################
## TIBCO network tuning #
#########################
net.core.rmem_default = 33554432
net.core.wmem_default = 33554432
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432Toggling the support.PIPE_MAX_SIZE to +32MB or temporarily removing these parameters mitigates the issue. Is there a better way of calculating support.PIPE_MAX_SIZE so it is reflective of the actual OS value? |
What does:
return? Note that the kernel buffer sizes above are, well, *really huge*. |
|
fcntl doesnt seem to like the parameter you mentioned - # cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.5 (Santiago)
# /local/0/opt/python-3.4.1/bin/python
Python 3.4.1 (default, Sep 24 2014, 12:23:21)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fcntl, os
>>> r, w = os.pipe()
>>> fcntl.fcntl(w, 1032)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
>>> |
|
Let's try with this instead:
|
|
With both the kernel parameters defined and undefined, I get the following output - # /local/0/opt/python-3.4.1/bin/python
Python 3.4.1 (default, Sep 29 2014, 13:31:39)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from socket import socket, SO_SNDBUF, SOL_SOCKET
>>> s = socket()
>>> s.getsockopt(SOL_SOCKET, SO_SNDBUF)
16384 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: