Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_socket failure on AIX #66591

Closed
DavidEdelsohn mannequin opened this issue Sep 12, 2014 · 9 comments
Closed

test_socket failure on AIX #66591

DavidEdelsohn mannequin opened this issue Sep 12, 2014 · 9 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@DavidEdelsohn
Copy link
Mannequin

DavidEdelsohn mannequin commented Sep 12, 2014

BPO 22397
Nosy @ncoghlan, @pitrou, @vstinner, @rbtcollins, @serhiy-storchaka
Files
  • 12958_aix.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2015-08-03.21:16:56.938>
created_at = <Date 2014-09-12.18:44:13.762>
labels = ['type-bug', 'library']
title = 'test_socket failure on AIX'
updated_at = <Date 2015-08-03.21:16:56.937>
user = 'https://bugs.python.org/DavidEdelsohn'

    bugs.python.org fields:

    activity = <Date 2015-08-03.21:16:56.937>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2015-08-03.21:16:56.938>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2014-09-12.18:44:13.762>
creator = 'David.Edelsohn'
dependencies = []
files = ['36612']
hgrepos = []
issue_num = 22397
keywords = ['patch']
message_count = 9.0
messages = ['226837', '227944', '227948', '228000', '228005', '228011', '247908', '247909', '247953']
nosy_count = 8.0
nosy_names = ['ncoghlan', 'pitrou', 'vstinner', 'baikie', 'rbcollins', 'python-dev', 'serhiy.storchaka', 'David.Edelsohn']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22397'
versions = ['Python 3.3', 'Python 3.4', 'Python 3.5']
    @DavidEdelsohn
    Copy link
    Mannequin Author

    DavidEdelsohn mannequin commented Sep 12, 2014

    AIX has the same test_socket problem with FDPassSeparate as Darwin in bpo-12958 so skip some tests.

    @DavidEdelsohn DavidEdelsohn mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 12, 2014
    @serhiy-storchaka
    Copy link
    Member

    LGTM.

    @serhiy-storchaka
    Copy link
    Member

    See also bpo-11192.

    @vstinner
    Copy link
    Member

    Current test_socket failures on PPC64 AIX 3.x buildbot:

    http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2672/steps/test/logs/stdio

    ======================================================================
    ERROR: testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 266, in _tearDown
    raise exc
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 278, in clientRun
    test_func()
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2778, in _testFDPassSeparateMinSpace
    array.array("i", [fd1]))]),
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1896, in sendmsgToServer
    *(args + self.sendmsg_to_server_defaults[len(args):]))
OSError: [Errno 22] Invalid argument

    ======================================================================
    ERROR: testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 266, in _tearDown
    raise exc
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 278, in clientRun
    test_func()
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2754, in _testFDPassSeparate
    array.array("i", [fd1]))]),
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1896, in sendmsgToServer
    *(args + self.sendmsg_to_server_defaults[len(args):]))
OSError: [Errno 22] Invalid argument

    ======================================================================
    ERROR: testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 266, in _tearDown
    raise exc
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 278, in clientRun
    test_func()
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2778, in _testFDPassSeparateMinSpace
    array.array("i", [fd1]))]),
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1896, in sendmsgToServer
    *(args + self.sendmsg_to_server_defaults[len(args):]))
OSError: [Errno 22] Invalid argument

    ======================================================================
    FAIL: testIPv4toString (test.test_socket.GeneralModuleTests)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 976, in testIPv4toString
    assertInvalid(f, '0.0.0.')
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 968, in <lambda>
    (OSError, ValueError), func, a
AssertionError: (<class 'OSError'>, <class 'ValueError'>) not raised by inet_aton

    ======================================================================
    FAIL: testIPv6toString (test.test_socket.GeneralModuleTests)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1033, in testIPv6toString
    assertInvalid('1:2:3:4:5:6:7:8:')
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1011, in <lambda>
    (OSError, ValueError), f, a
AssertionError: (<class 'OSError'>, <class 'ValueError'>) not raised by <lambda>

    ======================================================================
    FAIL: testFDPassSeparate (test.test_socket.RecvmsgSCMRightsStreamTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2742, in testFDPassSeparate
    maxcmsgs=2)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

    ======================================================================
    FAIL: testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2766, in testFDPassSeparateMinSpace
    maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

    ======================================================================
    FAIL: testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2742, in testFDPassSeparate
    maxcmsgs=2)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

    ======================================================================
    FAIL: testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2766, in testFDPassSeparateMinSpace
    maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

    Ran 528 tests in 35.259s

    FAILED (failures=6, errors=4, skipped=94)

    @vstinner
    Copy link
    Member

    Failing tests:

    • testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
    • testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
    • testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
    • testFDPassSeparate (test.test_socket.RecvmsgSCMRightsStreamTest)
    • testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
    • testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
    • testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)

    It looks like the failure come from code like:

    self.sendmsgToServer([MSG], [(socket.SOL_SOCKET,
                              socket.SCM_RIGHTS,
                              array.array("i", [fd0])),

    and

    self.doRecvmsg(self.serv_sock, len(MSG),
               socket.CMSG_SPACE(SIZEOF_INT) + socket.CMSG_LEN(SIZEOF_INT)),

    In clear, there are errors on tests sending file descriptors through UNIX sockets. So this issue is a duplicate of bpo-20718, but we may keep this one for AIX, and bpo-20718 for OpenBSD.

    It looks possible to send a file descriptor between processes but using a pipe with I_SENDFD and I_RECVFD ioctl. See:
    http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.commtrf2/I_SENDFD.htm
    http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.commtrf2/I_RECVFD.htm
    http://search.cpan.org/~fractal/AnyEvent-FDpasser-0.3.0/lib/AnyEvent/FDpasser.pm

    @vstinner
    Copy link
    Member

    The testFDPassSimple pass: it's possible to pass a file descriptor between two processes. Hum. The testMultipleFDPass test also pass: it's possible to pass multiple file descriptors. Both tests use SCM_RIGHTS. The testCMSG_SPACE test also pass.

    It looks like the difference between testMultipleFDPass and testFDPassSeparate is the number of "cmsgs" ("access ancillary data"): 1 cmsg for testMultipleFDPass, two for testFDPassSeparate.

    I don't understand if it is a bug in the Python socket.sendmsg() function serializing ancillary data, or "just" if the AIX kernel does not support multiple ancillary data for sendmsg().

    @david: any idea? Did you ask IBM?

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Aug 2, 2015

    New changeset e48826904e63 by Robert Collins in branch '3.4':
    Issue bpo-22397: Skip failing tests on AIX
    https://hg.python.org/cpython/rev/e48826904e63

    New changeset a12d29f87e4c by Robert Collins in branch '3.5':
    Issue bpo-22397: Skip failing tests on AIX
    https://hg.python.org/cpython/rev/a12d29f87e4c

    New changeset 62235755609f by Robert Collins in branch 'default':
    Issue bpo-22397: Skip failing tests on AIX
    https://hg.python.org/cpython/rev/62235755609f

    @rbtcollins
    Copy link
    Member

    I've updated the ref in the patch to this bug per Victor's comments, and applied.

    I'm not sure of protocol here. Do we close this ticket, or keep it open to analyze the actual cause?

    @rbtcollins rbtcollins changed the title test_socket failure on AIX test_socket not running all tests on AIX Aug 2, 2015
    @vstinner
    Copy link
    Member

    vstinner commented Aug 3, 2015

    I'm not interested to investigate the issue on AIX. IMHO the feature is
    just not supported by the kernel so there is nothing to investigate except
    if you want to hack AIX kernel.

    @vstinner vstinner changed the title test_socket not running all tests on AIX test_socket failure on AIX Aug 3, 2015
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @vstinner @bitdancer @rbtcollins @serhiy-storchaka