classification
Title: test_socket error on AIX
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.8, Python 3.7, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, Michael.Felt, alef, delhallt, giampaolo.rodola, ncoghlan, sable
Priority: normal Keywords: patch

Created on 2011-02-11 15:05 by sable, last changed 2019-06-19 14:46 by Michael.Felt. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8954 merged Michael.Felt, 2018-08-27 11:50
PR 9063 Michael.Felt, 2018-09-13 08:44
PR 14237 closed Michael.Felt, 2019-06-19 14:46
Messages (12)
msg128400 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-11 15:05
I have the following error in test_socket on AIX:

======================================================================
FAIL: testGetaddrinfo (__main__.GeneralModuleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./Lib/test/test_socket.py", line 640, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

----------------------------------------------------------------------
Ran 158 tests in 16.904s

FAILED (failures=1, skipped=4)
Traceback (most recent call last):
  File "./Lib/test/test_socket.py", line 2012, in <module>
    test_main()
  File "./Lib/test/test_socket.py", line 2008, in test_main
    support.run_unittest(*tests)
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py", line 1145, in run_unittest
    _run_suite(suite)
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py", line 1128, in _run_suite
    raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File "./Lib/test/test_socket.py", line 640, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

[119732 refs]

I haven't investigated yet.
msg128671 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-16 15:06
I am not sure this is directly related, but test_socket.py will hang forever on AIX 6.1 when this script is run as part of "make buildbottest".

It won't hang when running the script alone.
msg191132 - (view) Author: Delhallt (delhallt) Date: 2013-06-14 14:19
not a python problem, see closed aix issue http://www-01.ibm.com/support/docview.wss?uid=isg1IZ57712
msg191418 - (view) Author: alef (alef) Date: 2013-06-18 15:57
Same error with Python 2.7.3 on AIX 6.1
msg271277 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-07-25 14:41
FYI: test fails on AIX 5.3 TL7 SP0 (as expected. The IBM document refers to 5.3 TL8)

On AIX 6.1 TL9 SP5 (and probably earlier) the test passes. IMHO - can be closed (and README.AIX can be updated).

michael@x071:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]oslevel -q -s | head -5
Known Service Packs
-------------------
6100-09-05-1524
6100-09-04-1441

michael@x071:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]../../python test_socket.py
...
----------------------------------------------------------------------
Ran 124 tests in 15.860s

OK (skipped=7)

On AIX 5.3 (the expected fail...)
root@x064:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]oslevel -q -s | head -5
Known Service Packs
-------------------
5300-07-00-0000
5300-06-03-0732

root@x064:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]../../python test_socket.py

...

======================================================================
FAIL: testGetaddrinfo (__main__.GeneralModuleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_socket.py", line 691, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

----------------------------------------------------------------------
Ran 124 tests in 16.765s

FAILED (failures=1, skipped=7)
Traceback (most recent call last):
  File "test_socket.py", line 1804, in <module>
    test_main()
  File "test_socket.py", line 1800, in test_main
    test_support.run_unittest(*tests)
  File "/data/prj/aixtools/python/python-2.7.11.5/Lib/test/test_support.py", line 1428, in run_unittest
    _run_suite(suite)
  File "/data/prj/aixtools/python/python-2.7.11.5/Lib/test/test_support.py", line 1411, in _run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "test_socket.py", line 691, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1
msg310145 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-01-17 09:31
Back to master (python3-3.7)

(note: not your problem - but this link is gone: not a python problem, see closed aix issue http://www-01.ibm.com/support/docview.wss?uid=isg1IZ57712)

Getting this (obviously) on AIX 5.3, but also on AIX 6.1 (TL6), but not on AIX 7.1 TL2: (both gcc and xlc)

testGetSockOpt (test.test_socket.GeneralModuleTests) ... ok
testGetaddrinfo (test.test_socket.GeneralModuleTests) ... FAIL
testHostnameRes (test.test_socket.GeneralModuleTests) ... ok

However, what I get on all three systems (but do not see on the AIX buildbot running on AIX 7.2, and I have no access to AIX 7.2) is:

AIX 6.1:
Ran 669 tests in 25.899s

FAILED (failures=1, errors=114, skipped=119)

AIX 7.1:

Ran 669 tests in 25.917s

FAILED (errors=114, skipped=119)

What all the ERRORS have in common is:

  File "/data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py", line 2373, in setUp
    self.serv_sock.settimeout(self.fail_timeout)
AttributeError: 'SendmsgTests' object has no attribute 'serv_sock'

HTH - (and still digging)
msg310165 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-01-17 13:59
I have tried jumping into the pdp using

but do not come much further in understanding how the expectations expressed in these comments are satisfied:

 +2153  # Tests for the sendmsg()/recvmsg() interface.  Where possible, the
 +2154  # same test code is used with different families and types of socket
 +2155  # (e.g. stream, datagram), and tests using recvmsg() are repeated
 +2156  # using recvmsg_into().
 +2157  #
 +2158  # The generic test classes such as SendmsgTests and
 +2159  # RecvmsgGenericTests inherit from SendrecvmsgBase and expect to be
 +2160  # supplied with sockets cli_sock and serv_sock representing the
 +2161  # client's and the server's end of the connection respectively, and
 +2162  # attributes cli_addr and serv_addr holding their (numeric where
 +2163  # appropriate) addresses.
 +2164  #


++ Details ++
14 tests failed again:
    test_asyncio test_cmath test_ctypes test_datetime test_distutils
    test_httplib test_httpservers test_os test_posix test_socket
    test_ssl test_subprocess test_time test_utf8_mode


()
-> super().setUp()
(Pdb) s
--Call--
> /data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2187)setUp()
-> def setUp(self):
(Pdb) l
2182
2183        # Time in seconds to wait before considering a test failed, or
2184        # None for no timeout.  Not all tests actually set a timeout.
2185        fail_timeout = 3.0
2186
2187 ->     def setUp(self):
2188            self.misc_event = threading.Event()
2189            super().setUp()
2190
2191        def sendToServer(self, msg):
2192            # Send msg to the server.
(Pdb) s
> /data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2188)setUp()
-> self.misc_event = threading.Event()
(Pdb) self
<test.test_socket.RFC3542AncillaryTest testMethod=testCmsgTruncNoBufSize>
(Pdb) s
--Call--
> /data/prj/python/git/x067-python3-3.7/Lib/threading.py(499)__init__()
-> def __init__(self):
(Pdb) self
<threading.Event object at 0x3035f3d0>
(Pdb) dir(self)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_reset_internal_locks', 'clear', 'isSet', 'is_set', 'set', 'wait']
(Pdb) l
494
495         """
496
497         # After Tim Peters' event class (without is_posted())
498
499  ->     def __init__(self):
500             self._cond = Condition(Lock())
501             self._flag = False
502
503         def _reset_internal_locks(self):
504             # private!  called by Thread._reset_internal_locks by _after_fork()
(Pdb) n
> /data/prj/python/git/x067-python3-3.7/Lib/threading.py(500)__init__()
-> self._cond = Condition(Lock())
(Pdb) n
> /data/prj/python/git/x067-python3-3.7/Lib/threading.py(501)__init__()
-> self._flag = False
(Pdb) n
--Return--
> /data/prj/python/git/x067-python3-3.7/Lib/threading.py(501)__init__()->None
-> self._flag = False
(Pdb) n
> /data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2189)setUp()
-> super().setUp()
(Pdb) l
2184        # None for no timeout.  Not all tests actually set a timeout.
2185        fail_timeout = 3.0
2186
2187        def setUp(self):
2188            self.misc_event = threading.Event()
2189 ->         super().setUp()
2190
2191        def sendToServer(self, msg):
2192            # Send msg to the server.
2193            return self.cli_sock.send(msg)
2194
(Pdb) s
--Call--
> /data/prj/python/git/x067-python3-3.7/Lib/unittest/case.py(453)setUp()
-> def setUp(self):
(Pdb) l
448             called after tearDown on test failure or success.
449
450             Cleanup items are called even if setUp fails (unlike tearDown)."""
451             self._cleanups.append((function, args, kwargs))
452
453  ->     def setUp(self):
454             "Hook method for setting up the test fixture before exercising it."
455             pass
456
457         def tearDown(self):
458             "Hook method for deconstructing the test fixture after testing it."
(Pdb) n
> /data/prj/python/git/x067-python3-3.7/Lib/unittest/case.py(455)setUp()
-> pass
(Pdb) n
--Return--
> /data/prj/python/git/x067-python3-3.7/Lib/unittest/case.py(455)setUp()->None
-> pass
(Pdb) n
--Return--
> /data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2189)setUp()->None
-> super().setUp()
(Pdb) l
2184        # None for no timeout.  Not all tests actually set a timeout.
2185        fail_timeout = 3.0
2186
2187        def setUp(self):
2188            self.misc_event = threading.Event()
2189 ->         super().setUp()
2190
2191        def sendToServer(self, msg):
2192            # Send msg to the server.
2193            return self.cli_sock.send(msg)
2194
(Pdb) s
> /data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2375)setUp()
-> print('stop here') #@@@
(Pdb) l
2370
2371        def setUp(self):
2372            import pdb #@@@
2373            pdb.set_trace() #@@@
2374            super().setUp()
2375 ->         print('stop here') #@@@
2376            print(self)
2377            super().setUp()
2378            self.serv_sock.settimeout(self.fail_timeout)
2379
2380
(Pdb) dir(self)
['__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_addExpectedFailure', '_addSkip', '_addUnexpectedSuccess', '_baseAssertEqual', '_classSetupFailed', '_cleanup_lock', '_cleanups', '_deprecate', '_diffThreshold', '_feedErrorsToResult', '_formatMessage', '_getAssertEqualityFunc', '_outcome', '_subtest', '_testCmsgTruncNoBufSize', '_testMethodDoc', '_testMethodName', '_testOddCmsgSize', '_testRecvHopLimit', '_testRecvHopLimitCMSG_SPACE', '_testRecvTrafficClassAndHopLimit', '_testRecvTrafficClassAndHopLimitCMSG_SPACE', '_testSecomdCmsgTruncInData', '_testSecondCmsgTrunc0', '_testSecondCmsgTrunc1', '_testSecondCmsgTrunc2Int', '_testSecondCmsgTruncLen0Minus1', '_testSetHopLimit', '_testSetTrafficClassAndHopLimit', '_testSingleCmsgTrunc0', '_testSingleCmsgTrunc1', '_testSingleCmsgTrunc2Int', '_testSingleCmsgTruncInData', '_testSingleCmsgTruncLen0Minus1', '_truncateMessage', '_type_equality_funcs', 'addCleanup', 'addTypeEqualityFunc', 'ancillaryMapping', 'assertAlmostEqual', 'assertAlmostEquals', 'assertCountEqual', 'assertDictContainsSubset', 'assertDictEqual', 'assertEqual', 'assertEquals', 'assertFalse', 'assertGreater', 'assertGreaterEqual', 'assertIn', 'assertIs', 'assertIsInstance', 'assertIsNone', 'assertIsNot', 'assertIsNotNone', 'assertLess', 'assertLessEqual', 'assertListEqual', 'assertLogs', 'assertMultiLineEqual', 'assertNotAlmostEqual', 'assertNotAlmostEquals', 'assertNotEqual', 'assertNotEquals', 'assertNotIn', 'assertNotIsInstance', 'assertNotRegex', 'assertNotRegexpMatches', 'assertRaises', 'assertRaisesRegex', 'assertRaisesRegexp', 'assertRegex', 'assertRegexpMatches', 'assertSequenceEqual', 'assertSetEqual', 'assertTrue', 'assertTupleEqual', 'assertWarns', 'assertWarnsRegex', 'assert_', 'checkFlags', 'checkHopLimit', 'checkHopLimitTruncatedHeader', 'checkRecvmsgAddress', 'checkTrafficClassAndHopLimit', 'checkTruncatedSecondHeader', 'countTestCases', 'debug', 'defaultTestResult', 'doCleanups', 'doRecvmsg', 'fail', 'failIf', 'failIfAlmostEqual', 'failIfEqual', 'failUnless', 'failUnlessAlmostEqual', 'failUnlessEqual', 'failUnlessRaises', 'fail_timeout', 'failureException', 'hop_limit', 'id', 'longMessage', 'maxDiff', 'misc_event', 'msg_flags_common_set', 'msg_flags_common_unset', 'msg_flags_eor_indicator', 'msg_flags_non_eor_indicator', 'name', 'registerRecvmsgResult', 'run', 'sendToServer', 'sendmsgToServer', 'sendmsg_to_server_defaults', 'setUp', 'setUpClass', 'shortDescription', 'skipTest', 'subTest', 'tearDown', 'tearDownClass', 'testCmsgTruncNoBufSize', 'testOddCmsgSize', 'testRecvHopLimit', 'testRecvHopLimitCMSG_SPACE', 'testRecvTrafficClassAndHopLimit', 'testRecvTrafficClassAndHopLimitCMSG_SPACE', 'testSecomdCmsgTruncInData', 'testSecondCmsgTrunc0', 'testSecondCmsgTrunc1', 'testSecondCmsgTrunc2Int', 'testSecondCmsgTruncLen0Minus1', 'testSetHopLimit', 'testSetTrafficClassAndHopLimit', 'testSingleCmsgTrunc0', 'testSingleCmsgTrunc1', 'testSingleCmsgTrunc2Int', 'testSingleCmsgTruncInData', 'testSingleCmsgTruncLen0Minus1', 'traffic_class']
(Pdb) self
<test.test_socket.RFC3542AncillaryTest testMethod=testCmsgTruncNoBufSize>
msg323533 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-08-14 19:05
The original error reported was fixed by AIX - in AIX 6.1.

It will remain unresolved in AIX 5.3.

There are currently two other errors in test_socket.

FAIL: test_getnameinfo_ipv6_scopeid_symbolic (test.test_socket.GeneralModuleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.7/Lib/test/test_socket.py", line 1649, in test_getnameinfo_ipv6_scopeid_symbolic
    self.assertEqual(nameinfo, ('ff02::1de:c0:face:8d%' + test_interface, '1234'))
AssertionError: Tuples differ: ('ff02::1de:c0:face:8d', '1234') != ('ff02::1de:c0:face:8d%en0', '1234')

First differing element 0:
'ff02::1de:c0:face:8d'
'ff02::1de:c0:face:8d%en0'

- ('ff02::1de:c0:face:8d', '1234')
+ ('ff02::1de:c0:face:8d%en0', '1234')
?                       ++++

I assume the capital D in the argument in the test:
        sockaddr = ('ff02::1de:c0:face:8D', 1234, 0, ifindex)  # Note capital letter `D`.

is meant to trigger getting the IPv6 zoneinfo based on ifindex.

AIX does have a routine to strip zoneinfo off an IPv6 address, but the getnameinfo() and getaddrinfo() do not support this.

I can create a new issue specifically for IPv6 zone id issue, or I can add a PR here. 

Please advise (the PR is merely adding "skipUnless" logic to test_socket.py.

ERROR: test_getaddrinfo_ipv6_scopeid_symbolic
msg323953 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-08-23 15:05
I nearly have a patch - to resolve all the remaining test_socket() issues with AIX. However, this (new?) issue remains:

        f = lambda a: inet_pton(AF_INET6, a)
        assertInvalid = lambda a: self.assertRaises(
            (OSError, ValueError), f, a
        )

>>> f('1:2:3:4:5:6:7:8')
b'\x00\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06\x00\x07\x00\x08'
>>> f('1:2:3:4:5:6:7:8:')
b'\x00\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06\x00\x07\x00\x08'
>>> f('1::')
b'\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
>>> f('1:2:3:4:5:6:7:8::')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
OSError: illegal IP address string passed to inet_pton

>>> f('1:2:3:4:5:6:7')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
OSError: illegal IP address string passed to inet_pton


Is, maybe, the last test meant to be '::' on the end, rather than ':'?
msg332527 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-12-26 03:34
New changeset 5661459f5f508ea4bd24c13cbc861543f283147e by Nick Coghlan (Michael Felt) in branch 'master':
bpo-11192: Skip unsupported cases in test_socket on AIX (GH-8954)
https://github.com/python/cpython/commit/5661459f5f508ea4bd24c13cbc861543f283147e
msg337634 - (view) Author: Michael Felt (Michael.Felt) * Date: 2019-03-10 18:40
Could this also be backported to Version 3.7 and 3.6 (I do not expect it to be backported to 2.7, but I had mistakenly removed it 2.7 when I changed it to 3.8 - and should have added 3.6 and 3.7 then).
msg337699 - (view) Author: Michael Felt (Michael.Felt) * Date: 2019-03-11 21:43
On 10/03/2019 19:40, Michael Felt wrote:
> Michael Felt <aixtools@felt.demon.nl> added the comment:
>
> Could this also be backported to Version 3.7 and 3.6 (I do not expect it to be backported to 2.7, but I had mistakenly removed it 2.7 when I changed it to 3.8 - and should have added 3.6 and 3.7 then).
Likewise: As 3.6 is in security mode - I guess only 3.7 then.
>
> ----------
> versions: +Python 2.7, Python 3.6, Python 3.7
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue11192>
> _______________________________________
>
History
Date User Action Args
2019-06-19 14:46:58Michael.Feltsetpull_requests: + pull_request14073
2019-03-12 07:35:56Michael.Feltsetversions: - Python 3.6
2019-03-11 21:43:33Michael.Feltsetmessages: + msg337699
2019-03-10 18:40:56Michael.Feltsetmessages: + msg337634
versions: + Python 2.7, Python 3.6, Python 3.7
2018-12-28 14:08:27ncoghlansetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-12-26 03:34:41ncoghlansetnosy: + ncoghlan
messages: + msg332527
2018-09-13 08:44:23Michael.Feltsetpull_requests: + pull_request8686
2018-09-11 06:00:04Michael.Feltsettype: behavior
components: + Tests
versions: + Python 3.8, - Python 3.7
2018-08-27 11:50:35Michael.Feltsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8430
2018-08-23 15:05:23Michael.Feltsetmessages: + msg323953
2018-08-14 19:05:38Michael.Feltsetmessages: + msg323533
2018-01-17 13:59:56Michael.Feltsetmessages: + msg310165
2018-01-17 09:31:50Michael.Feltsetmessages: + msg310145
versions: + Python 3.7, - Python 2.7, Python 3.2
2016-07-25 14:41:14Michael.Feltsetnosy: + Michael.Felt
messages: + msg271277
2013-06-18 16:00:53David.Edelsohnsetnosy: + David.Edelsohn
2013-06-18 15:57:28alefsetmessages: + msg191418
versions: + Python 2.7
2013-06-14 14:19:40delhalltsetnosy: + delhallt
messages: + msg191132
2013-03-14 13:36:05alefsetnosy: + alef
2011-02-16 15:06:35sablesetmessages: + msg128671
2011-02-11 15:09:11pitrousetnosy: + giampaolo.rodola
2011-02-11 15:05:50sablecreate