classification
Title: test_os.test_trailers() is failing on AMD64 FreeBSD 9.0 dtrace 3.x
Type: Stage:
Components: Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: koobs, loewis, neologix, python-dev, vstinner
Priority: normal Keywords: patch

Created on 2013-06-25 00:25 by vstinner, last changed 2013-08-16 13:40 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
sendfile_trailers.patch vstinner, 2013-08-13 23:13 review
Messages (7)
msg191827 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-06-25 00:25
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%203.x/builds/1560/steps/test/logs/stdio

======================================================================
FAIL: test_trailers (test.test_os.TestSendfile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd/build/Lib/test/test_os.py", line 1962, in test_trailers
    self.assertEqual(data, b"abcde12345")
AssertionError: b'abcde\x00\x00\x00...\x00\x0012345' != b'abcde12345'
Warning -- threading._dangling was modified by test_os
test test_os failed

(I replaced a long list of \x00 with "...")

I don't understand why test_trailers() of test_os tries to copy 4096 bytes using os.sendto(), whereas the input file only contains 5 bytes.
msg191858 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2013-06-25 13:49
I recently updated the buildbot host to the latest 9-STABLE sources (changeset delta is approximately 4-6 weeks)

On a hunch I asked our src committers if there had been any sendfile related changes and was pointed to 3 changesets that were merged from current (MFC) to stable/9 4 weeks ago:

http://svnweb.freebsd.org/base?view=revision&revision=250907

I reverted the kernel to r250906 (hunch commit -1) that resulted in a clean run:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%203.x/builds/1563

Manual Rebuild: Trying to isolate test_trailing with older kernel

I have it on preliminary evidence that our regression tests are passing on this functionality, but analysis at the Python end for a root cause and details on how the test implementation plays a role would be helpful.
msg194269 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2013-08-03 16:19
The test shouldn't pass 4096 as nbytes: apparently, recent FreeBSD kernels zero-fill.
msg195102 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-08-13 23:13
@koobs: The problem is in the Python test, not in FreeBSD. Can you upgrade the kernel to the last development version?

Attached patch should fix the test.
msg195242 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2013-08-15 09:40
As per our IRC conversation, our 'koobs-freebsd10' bot also reproduces the failure and can be used to test the patch.
msg195243 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-08-15 09:58
New changeset 924d327da3af by Victor Stinner in branch '3.3':
Issue #18296: Try to fix TestSendfile.test_trailers() of test_os on FreeBSD
http://hg.python.org/cpython/rev/924d327da3af

New changeset 92039fb68483 by Victor Stinner in branch 'default':
(Merge 3.3) Issue #18296: Try to fix TestSendfile.test_trailers() of test_os on FreeBSD
http://hg.python.org/cpython/rev/92039fb68483
msg195332 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-08-16 13:40
The test does now pass with the patch, so I close the issue.
History
Date User Action Args
2013-08-16 13:40:49vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg195332
2013-08-15 09:58:05python-devsetnosy: + python-dev
messages: + msg195243
2013-08-15 09:40:36koobssetmessages: + msg195242
2013-08-13 23:13:39vstinnersetfiles: + sendfile_trailers.patch
keywords: + patch
messages: + msg195102
2013-08-03 16:19:56neologixsetnosy: + neologix
messages: + msg194269
2013-06-25 13:49:14koobssetmessages: + msg191858
2013-06-25 09:35:18koobssetnosy: + koobs
2013-06-25 08:21:10serhiy.storchakasetnosy: + loewis
2013-06-25 00:25:42vstinnercreate