classification
Title: Solaris ctypes_test failures
Type: behavior Stage:
Components: Tests Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, gco, jcea, meador.inge, trent, vstinner, yippi
Priority: normal Keywords: patch

Created on 2012-12-19 23:24 by yippi, last changed 2018-05-09 22:49 by gco.

Files
File name Uploaded Description Edit
ctypes.out yippi, 2012-12-19 23:24 python program and output showing failures
0001-Fix-ctypes-bitfield-test-code-to-account-for-compile.patch gco, 2018-05-09 00:45
Messages (6)
msg177792 - (view) Author: yippi (yippi) Date: 2012-12-19 23:24
Note I am building Python 3.3 with the Sun Studio compiler on Solaris 11.  When I run the tests, I get these 2 ctypes errors:

======================================================================
FAIL: test_ints (ctypes.test.test_bitfields.C_Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/bacamero/ul/components/python/python33/Python-3.Lib/ctypes/test/test_bitfields.py", line 40, in test_ints
    self.assertEqual(getattr(b, name), func(byref(b), name.encode('ascii')))
AssertionError: -1 != 1
======================================================================
FAIL: test_shorts (ctypes.test.test_bitfields.C_Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/bacamero/ul/components/python/python33/Python-3.Lib/ctypes/test/test_bitfields.py", line 47, in test_shorts
    self.assertEqual(getattr(b, name), func(byref(b), name.encode('ascii')))
AssertionError: -32 != 32
----------------------------------------------------------------------

I see almost identical errors in Python 2.6 and 2.7.  The same two tests fail, though the format of the error message is a bit different.  For example it looks like this in 2.7:
AssertionError: Tuples differ: ('A', 1, -1) != ('A', 1, 1)
AssertionError: Tuples differ: ('R', 32, -32) != ('R', 32, 32)

I made a simple standalone test program that prints out all the values
instead of stopping on first failure.  The script and output is attached
for reference. 

I think this might be related to issue #16275.  On Solaris 11 we are building Python with the configure --with -system-ffi argument.  On Solaris 11, libffi 3.0.9 is used.  I tried updating libffi to the latest 3.0.11 version and rebuilding, but that did not help.

Any pointers about how to debug this would be helpful.  It is not clear to me if this is a libffi issue, a compiler issue, a problem in Python or something else.  Is there any additional information I could provide to help track this down?
msg206342 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-12-16 18:38
The ctypes issue also affects test_uuid. #19045 is a duplicate of this bug, too.

======================================================================
FAIL: test_ints (ctypes.test.test_bitfields.C_Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/ctypes/test/test_bitfields.py", line 40, in test_ints
    self.assertEqual(getattr(b, name), func(byref(b), name.encode('ascii')))
AssertionError: -1 != 1

======================================================================
FAIL: test_shorts (ctypes.test.test_bitfields.C_Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/ctypes/test/test_bitfields.py", line 47, in test_shorts
    self.assertEqual(getattr(b, name), func(byref(b), name.encode('ascii')))
AssertionError: -32 != 32

======================================================================
FAIL: test_uuid4 (test.test_uuid.TestUUID)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/test/test_uuid.py", line 441, in test_uuid4
    equal(u.variant, uuid.RFC_4122)
AssertionError: 'reserved for Microsoft compatibility' != 'specified in RFC 4122'
- reserved for Microsoft compatibility
+ specified in RFC 4122
msg211499 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-02-18 09:08
Any progress on this issue? There are still failures on Solaris buildbots.
msg223813 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-24 07:30
Only one Solaris box was online when I looked and that had passed ctypes tests, but surely this will have been sorted out by now?
msg316300 - (view) Author: Greg Onufer (gco) Date: 2018-05-09 00:45
The bitfields failures are due to Python not implementing bitfields the same as the compiler.

https://docs.oracle.com/cd/E77782_01/html/E77792/gqexw.html

"Signed and Unsigned int Bit-fields
Bit-fields which are declared as int (not signed int or unsigned int) can be implemented by the compiler using either signed or unsigned types. This makes a difference when extracting a value and deciding whether to sign extend it.

The Oracle Developer Studio compiler uses unsigned types for int bit-fields and the gcc compiler uses signed types. Use the gcc –funsigned-bitfields flag to control this behavior.

For more information, see the sixth list item at https://gcc.gnu.org/onlinedocs/gcc-3.3.6/gcc/Non_002dbugs.html."

When the test compares the ctypes extraction of the bitfield (signed) and the compiler-native extraction of the bitfield (unsigned), they miscompare if the high-bit of the field is set.

If Python wants bitfields extracted from signed integral types to be signed, the C code in the test case needs to account for the compiler implementation defined behavior and sign extend the bitfield before returning it.

See patch in attachment.  The bitfield tests pass with those changes.
msg316341 - (view) Author: Greg Onufer (gco) Date: 2018-05-09 22:49
Or probably just change the type of the bitfield from "int" to "signed int"!
History
Date User Action Args
2018-05-09 22:49:51gcosetmessages: + msg316341
2018-05-09 11:43:25BreamoreBoysetnosy: - BreamoreBoy
2018-05-09 00:45:38gcosetfiles: + 0001-Fix-ctypes-bitfield-test-code-to-account-for-compile.patch

nosy: + gco
messages: + msg316300

keywords: + patch
2014-07-24 07:30:12BreamoreBoysetnosy: + BreamoreBoy

messages: + msg223813
versions: + Python 3.5, - Python 3.3
2014-02-18 09:08:59vstinnersetnosy: + vstinner
messages: + msg211499
2013-12-16 18:38:14christian.heimessetnosy: + christian.heimes

messages: + msg206342
versions: + Python 3.4, - Python 2.6
2013-01-01 16:48:18meador.ingesetnosy: + meador.inge
2012-12-20 08:37:53trentsetnosy: + trent
2012-12-19 23:43:45jceasetnosy: + jcea
2012-12-19 23:24:47yippicreate