Author koobs
Recipients koobs, pablogsal
Date 2018-09-27.10:01:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1538042509.94.0.545547206417.issue34521@psf.upfronthosting.co.za>
In-reply-to
Content
Copying in my original email to Pablo that elucidated the test failure cause and resolution, including upstream commit/review/issue references for the recvmsg(2) changes IN CURRENT.

Full buildbot worker log attached

While I'm here, update list of versions (branches) affected.

------------------

The test suite fails:

  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/multiprocessing/reduction.py",
line 164, in recvfds
    raise RuntimeError('received %d items of ancdata' %
RuntimeError: received 0 items of ancdata

For these two tests (further up in the log [1]) ...

test_fd_transfer
(test.test_multiprocessing_forkserver.WithProcessesTestConnection) ... FAIL
test_large_fd_transfer
(test.test_multiprocessing_forkserver.WithProcessesTestConnection) ... FAIL

Further up we see (in the initial test run) ...

4 tests failed:
    test_multiprocessing_fork test_multiprocessing_forkserver
    test_multiprocessing_spawn test_socket

Then further up we see:

======================================================================
FAIL: testFDPassSeparateMinSpace
(test.test_socket.RecvmsgSCMRightsStreamTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3186, in testFDPassSeparateMinSpace
    self.checkRecvmsgFDs(2,
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3107, in checkRecvmsgFDs
    self.assertEqual(len(fds), numfds)
AssertionError: 1 != 2
======================================================================
FAIL: testFDPassSeparateMinSpace
(test.test_socket.RecvmsgIntoSCMRightsStreamTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3186, in testFDPassSeparateMinSpace
    self.checkRecvmsgFDs(2,
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3107, in checkRecvmsgFDs
    self.assertEqual(len(fds), numfds)
AssertionError: 1 != 2
----------------------------------------------------------------------
Ran 554 tests in 27.636s
FAILED (failures=2, skipped=57)

I looked for recvmsg() changes in FreeBSD recently, and found ...

Commit: https://svnweb.freebsd.org/base?view=revision&revision=337423
Review: https://reviews.freebsd.org/D16561
Bug ID: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=131876
Bug Desc:[socket] FD leak by receiving SCM_RIGHTS by recvmsg with small
control message buffer

The commit/bug/review details above point to recvmsg specific changes,
specifically, but maybe not *just*, for SCM_RIGHTS messages, which (in
my opinion) is way too similar to the test case specifics not to be related.

Also, the commit date of Aug 7 16:36:48 2018  seems right around the
time I updated the machine and the tests began to fail.

I can't say yet whether or not a bug crept in with the change, *but*,
the commit does say that it *fixes* bugs, which the python tests, or the
socket code in Python may be have been relying on.

It may also be that FreeBSD specific-behaviour isn't being
handled by the code and/or tests (truncated now where it wasn't before?)

It could also be the buffer sizes the socket code/tests use.

------------------
History
Date User Action Args
2018-09-27 10:01:52koobssetrecipients: + koobs, pablogsal
2018-09-27 10:01:49koobssetmessageid: <1538042509.94.0.545547206417.issue34521@psf.upfronthosting.co.za>
2018-09-27 10:01:49koobslinkissue34521 messages
2018-09-27 10:01:44koobscreate