Title: test_socket testCmsgTruncLen0 gets "received malformed or improperly-truncated ancillary data" warning under OS X
Type: Stage: patch review
Components: macOS, Tests Versions: Python 3.11, Python 3.10, Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: baikie, brett.cannon, iritkatriel, ned.deily, pablogsal, ronaldoussoren
Priority: normal Keywords: patch

Created on 2015-03-31 16:55 by brett.cannon, last changed 2021-07-31 18:28 by iritkatriel.

Pull Requests
URL Status Linked Edit
PR 27512 closed iritkatriel, 2021-07-31 18:25
Messages (5)
msg239732 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-03-31 16:55
ERROR: testCmsgTruncLen0 (__main__.RecvmsgSCMRightsStreamTest)
Traceback (most recent call last):
  File "/Users/bcannon/Repositories/cpython/pep-488/Lib/test/", line 2956, in testCmsgTruncLen0
    self.checkTruncatedArray(ancbuf=socket.CMSG_LEN(0), maxdata=0)
  File "/Users/bcannon/Repositories/cpython/pep-488/Lib/test/", line 2937, in checkTruncatedArray
    len(MSG), ancbuf)
  File "/Users/bcannon/Repositories/cpython/pep-488/Lib/test/", line 1921, in doRecvmsg
    result = sock.recvmsg(bufsize, *args)
RuntimeWarning: received malformed or improperly-truncated ancillary data
msg239762 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-03-31 23:28
Original title: "test_socket fails under -Werror"

These are long-time, normally non-fatal warnings when running test_socket on OS X with Python 3 independent of -Werror, introduced with the socket sendmsg() and recvmsg() feature in the changes for Issue6560 and a followup in Issue12958.  I guess we haven't followed up on them since.  I note that there is now an open issue (Issue20669) about a set of test_socket failures under OpenBSD including this case.  So perhaps this test behavior also has its origins in differences between BSD vs others, further complicated by subsequent divergence of the OS X network layer from its BSD origins.

Interestingly, because the warning does not normally cause a test failure (that is, without the Python -Werror option as used in this issue), the warning messages do not show up in the OS X buildbot logs at all because the buildbot test runs use the -m test (test.regrtest) -W option (not to be confused with the Python -W option).  Without -m test -W, the warnings do appear but, without Python -Werror, cause no failures.
msg241246 - (view) Author: David Watson (baikie) Date: 2015-04-16 17:36
In this test (and nearby ones), the receiver provides too little space to receive the file descriptors that are sent, so that the control message has to be truncated if it is to be received at all.  The warning may indicate that the cmsghdr struct received still claims that the message contains the full amount of data (so that socket.recvmsg() would overrun the buffer if it actually believed the cmsghdr).

With that in mind, you might want to check whether file descriptors that don't fit are still received (say the sender sends three file descriptors when the receiver only asks for two, and the receiver gets a control message saying it now has descriptors on numbers 3 and 4, but it also gets a file descriptor on number 5 which it isn't told about).  If they are, it's a kernel bug which could be used for denial of service by exhausting the receiving process' quota of file descriptors (and the warning would probably go away if it were fixed).
msg328692 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-28 15:17
Duplicate Issue35092 notes that running the tests with -Werror turns these warnings into test failures.
msg398659 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-31 18:28
In PR27512 I tried to make the test expect the warning, but then it fails on ubuntu, so this is not the right solution.
Date User Action Args
2021-07-31 18:28:02iritkatrielsetmessages: + msg398659
2021-07-31 18:25:39iritkatrielsetkeywords: + patch
nosy: + iritkatriel

pull_requests: + pull_request26031
stage: needs patch -> patch review
2021-07-31 18:24:31iritkatrielsetversions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.6, Python 3.7, Python 3.8
2018-10-28 15:46:07pablogsalsetnosy: + pablogsal
2018-10-28 15:17:33ned.deilysetversions: + Python 3.6, Python 3.7, Python 3.8, - Python 3.4, Python 3.5
nosy: + ronaldoussoren

messages: + msg328692

components: + macOS
2018-10-28 15:16:34ned.deilylinkissue35092 superseder
2015-04-16 17:36:04baikiesetnosy: + baikie
messages: + msg241246
2015-03-31 23:28:19ned.deilysetnosy: + ned.deily
title: test_socket fails under -Werror -> test_socket testCmsgTruncLen0 gets "received malformed or improperly-truncated ancillary data" warning under OS X
messages: + msg239762

versions: + Python 3.4
2015-03-31 16:55:36brett.cannoncreate