classification
Title: test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ... Fatal Python error: Bus error
Type: crash Stage: resolved
Components: Interpreter Core, Tests Versions: Python 3.4, Python 3.5
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, iritkatriel, jcea, mark.dickinson, petriborg, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2015-03-26 13:47 by petriborg, last changed 2021-06-16 20:30 by petriborg. This issue is now closed.

Files
File name Uploaded Description Edit
fnv_memcpy.patch vstinner, 2015-03-27 10:40 review
Messages (19)
msg239330 - (view) Author: Peter (petriborg) Date: 2015-03-26 13:47
I compiled Python 3.4.3 on Solaris 11, and when I ran the regression test, I get a core dump when the hash() function was being used. I went through the bug database looking for something similar but couldn't find anything.

Tests like:
test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ... Fatal Python error: Bus error

Current thread 0x00000001 (most recent call first):
  File "/usr/local/lib/python3.4/test/test_hash.py", line 89 in test_unaligned_buffers
  File "/usr/local/lib/python3.4/unittest/case.py", line 577 in run
  File "/usr/local/lib/python3.4/unittest/case.py", line 625 in __call__
  File "/usr/local/lib/python3.4/unittest/suite.py", line 122 in run
  File "/usr/local/lib/python3.4/unittest/suite.py", line 84 in __call__
  File "/usr/local/lib/python3.4/unittest/suite.py", line 122 in run
  File "/usr/local/lib/python3.4/unittest/suite.py", line 84 in __call__
  File "/usr/local/lib/python3.4/unittest/suite.py", line 122 in run
  File "/usr/local/lib/python3.4/unittest/suite.py", line 84 in __call__
  File "/usr/local/lib/python3.4/unittest/runner.py", line 168 in run
  File "/usr/local/lib/python3.4/test/support/__init__.py", line 1769 in _run_suite
  File "/usr/local/lib/python3.4/test/support/__init__.py", line 1803 in run_unittest
  File "/usr/local/lib/python3.4/test/regrtest.py", line 1279 in test_runner
  File "/usr/local/lib/python3.4/test/regrtest.py", line 1280 in runtest_inner
  File "/usr/local/lib/python3.4/test/regrtest.py", line 978 in runtest
  File "/usr/local/lib/python3.4/test/regrtest.py", line 763 in main
  File "/usr/local/lib/python3.4/test/regrtest.py", line 1564 in main_in_temp_cwd
  File "/usr/local/lib/python3.4/test/__main__.py", line 3 in <module>
  File "/usr/local/lib/python3.4/runpy.py", line 85 in _run_code
  File "/usr/local/lib/python3.4/runpy.py", line 170 in _run_module_as_main
Bus Error (core dumped)


and


test_hash_equality (test.datetimetester.TestTime_Fast) ... Fatal Python error: Bus error

Current thread 0x00000001 (most recent call first):
  File "/usr/local/lib/python3.4/test/datetimetester.py", line 2155 in test_hash_equality
  File "/usr/local/lib/python3.4/unittest/case.py", line 577 in run
  File "/usr/local/lib/python3.4/unittest/case.py", line 625 in __call__
  File "/usr/local/lib/python3.4/unittest/suite.py", line 122 in run
  File "/usr/local/lib/python3.4/unittest/suite.py", line 84 in __call__
  File "/usr/local/lib/python3.4/unittest/suite.py", line 122 in run
  File "/usr/local/lib/python3.4/unittest/suite.py", line 84 in __call__
  File "/usr/local/lib/python3.4/unittest/runner.py", line 168 in run
  File "/usr/local/lib/python3.4/test/support/__init__.py", line 1769 in _run_suite
  File "/usr/local/lib/python3.4/test/support/__init__.py", line 1803 in run_unittest
  File "/usr/local/lib/python3.4/test/test_datetime.py", line 45 in test_main
  File "/usr/local/lib/python3.4/test/regrtest.py", line 1280 in runtest_inner
  File "/usr/local/lib/python3.4/test/regrtest.py", line 978 in runtest
  File "/usr/local/lib/python3.4/test/regrtest.py", line 763 in main
  File "/usr/local/lib/python3.4/test/regrtest.py", line 1564 in main_in_temp_cwd
  File "/usr/local/lib/python3.4/test/__main__.py", line 3 in <module>
  File "/usr/local/lib/python3.4/runpy.py", line 85 in _run_code
  File "/usr/local/lib/python3.4/runpy.py", line 170 in _run_module_as_main
Bus Error (core dumped)



I then ran the same test through gdb:
$ gdb /usr/local/bin/python3.4
GNU gdb (GDB) 7.8.1
<snip>
(gdb) run -m test -v test_hash
Starting program: /usr/local/bin/python3.4 -m test -v test_hash
[Thread debugging using libthread_db enabled]
[New Thread 1 (LWP 1)]
== CPython 3.4.3 (default, Mar 25 2015, 17:35:25) [GCC 4.6.2]
==   Solaris-2.11-sun4v-sparc-32bit-ELF big-endian
==   hash algorithm: fnv 32bit
==   /tmp/test_python_12329
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_
site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_hash
test_empty_string (test.test_hash.BytesHashRandomizationTests) ... ok
test_fixed_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_long_fixed_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_null_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.DatetimeDateTests) ... ok
test_randomized_hash (test.test_hash.DatetimeDatetimeTests) ... FAIL
test_randomized_hash (test.test_hash.DatetimeTimeTests) ... FAIL
test_hashes (test.test_hash.HashBuiltinsTestCase) ... ok
test_hash_distribution (test.test_hash.HashDistributionTestCase) ... ok
test_coerced_floats (test.test_hash.HashEqualityTestCase) ... ok
test_coerced_integers (test.test_hash.HashEqualityTestCase) ... ok
test_numeric_literals (test.test_hash.HashEqualityTestCase) ... ok
test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ...
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0xff25d14c in fnv (src=0x1d2289, len=127) at Python/pyhash.c:267
267             x = (_PyHASH_MULTIPLIER * x) ^ block.value;
(gdb) bt
#0  0xff25d14c in fnv (src=0x1d2289, len=127) at Python/pyhash.c:267
#1  0xff25d3f8 in _Py_HashBytes (src=0x1d2289, len=127) at Python/pyhash.c:186
#2  0xff1a7ce4 in memory_hash (self=0xfdff5570) at Objects/memoryobject.c:2793
#3  0xff1afa40 in PyObject_Hash (v=0xfdff5570) at Objects/object.c:757
#4  0xff22b6fc in builtin_hash (self=0xfee23600, v=0xfdff5570) at Python/bltinmodule.c:1269
#5  0xff236a70 in call_function (oparg=<optimized out>, pp_stack=0xffbfcc94) at Python/ceval.c:4224
#6  PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2838
#7  0xff237790 in fast_function (nk=<optimized out>, na=<optimized out>, n=1, pp_stack=0xffbfcd8c,
    func=<optimized out>) at Python/ceval.c:4334
<snip>


(gdb) list
262
263         x = (Py_uhash_t) _Py_HashSecret.fnv.prefix;
264         x ^= (Py_uhash_t) *p << 7;
265         while (blocks--) {
266             PY_UHASH_CPY(block.bytes, p);
267             x = (_PyHASH_MULTIPLIER * x) ^ block.value;
268             p += SIZEOF_PY_UHASH_T;
269         }
270         /* add remainder */
271         for (; remainder > 0; remainder--)



Further info:
$ uname -a
SunOS vstrlily1d 5.11 11.1 sun4v sparc sun4v Solaris

$ gcc --verbose
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/sparc-sun-solaris2.11/4.6.2/lto-wrapper
Target: sparc-sun-solaris2.11
Configured with: ../gcc-4.6.2/configure --prefix=/usr/local --enable-languages=c,c++ --disable-nls --with-gnu-as --with-gnu-ld --target=sparc-sun-solaris2.11
Thread model: posix
gcc version 4.6.2 (GCC)
msg239347 - (view) Author: Peter (petriborg) Date: 2015-03-26 17:57
I went and recompiled with:
$ ./configure --prefix=/usr/local --enable-shared --with-hash-algorithm=siphash24

But this crashed as well.

test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ... Fatal Python error: Bus error

Current thread 0x00000001 (most recent call first):
  File "/usr/local/src/Python-3.4.3/Lib/test/test_hash.py", line 89 in test_unaligned_buffers
  File "/usr/local/src/Python-3.4.3/Lib/unittest/case.py", line 577 in run
  File "/usr/local/src/Python-3.4.3/Lib/unittest/case.py", line 625 in __call__
  File "/usr/local/src/Python-3.4.3/Lib/unittest/suite.py", line 122 in run
  File "/usr/local/src/Python-3.4.3/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/local/src/Python-3.4.3/Lib/unittest/suite.py", line 122 in run
  File "/usr/local/src/Python-3.4.3/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/local/src/Python-3.4.3/Lib/unittest/suite.py", line 122 in run
  File "/usr/local/src/Python-3.4.3/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/local/src/Python-3.4.3/Lib/unittest/runner.py", line 168 in run
  File "/usr/local/src/Python-3.4.3/Lib/test/support/__init__.py", line 1769 in _run_suite
  File "/usr/local/src/Python-3.4.3/Lib/test/support/__init__.py", line 1803 in run_unittest
  File "/usr/local/src/Python-3.4.3/Lib/test/regrtest.py", line 1279 in test_runner
  File "/usr/local/src/Python-3.4.3/Lib/test/regrtest.py", line 1280 in runtest_inner
  File "/usr/local/src/Python-3.4.3/Lib/test/regrtest.py", line 978 in runtest
  File "/usr/local/src/Python-3.4.3/Lib/test/regrtest.py", line 763 in main
  File "/usr/local/src/Python-3.4.3/Lib/test/regrtest.py", line 1564 in main_in_temp_cwd
  File "/usr/local/src/Python-3.4.3/Lib/test/__main__.py", line 3 in <module>
  File "/usr/local/src/Python-3.4.3/Lib/runpy.py", line 85 in _run_code
  File "/usr/local/src/Python-3.4.3/Lib/runpy.py", line 170 in _run_module_as_main
Bus Error (core dumped)


test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ...
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0xff25d3d8 in siphash24 (src=0x1d2809, src_sz=<optimized out>) at Python/pyhash.c:387
387             PY_UINT64_T mi = _le64toh(*in);
(gdb) bt
#0  0xff25d3d8 in siphash24 (src=0x1d2809, src_sz=<optimized out>) at Python/pyhash.c:387
#1  0xff25dfa0 in _Py_HashBytes (src=0x1d2809, len=127) at Python/pyhash.c:186
#2  0xff1a7ce4 in memory_hash (self=0xfdfc5dc0) at Objects/memoryobject.c:2793
#3  0xff1afa40 in PyObject_Hash (v=0xfdfc5dc0) at Objects/object.c:757
#4  0xff22b6fc in builtin_hash (self=0xfee23600, v=0xfdfc5dc0) at Python/bltinmodule.c:1269
#5  0xff236a70 in call_function (oparg=<optimized out>, pp_stack=0xffbfcd64) at Python/ceval.c:4224
#6  PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2838
#7  0xff237790 in fast_function (nk=<optimized out>, na=<optimized out>, n=1, pp_stack=0xffbfce5c,
    func=<optimized out>) at Python/ceval.c:4334


(gdb) list
382         PY_UINT64_T t;
383         PY_UINT8_T *pt;
384         PY_UINT8_T *m;
385
386         while (src_sz >= 8) {
387             PY_UINT64_T mi = _le64toh(*in);
388             in += 1;
389             src_sz -= 8;
390             v3 ^= mi;
391             DOUBLE_ROUND(v0,v1,v2,v3);
msg239348 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-26 18:18
Siphash24 implementation is not designed to work on platforms that require aligned access. But I'm surprised that fnv implementation crashes. I don't see anything wrong. May be gcc needs some special options to produce correct binaries on this platform?
msg239351 - (view) Author: Peter (petriborg) Date: 2015-03-26 19:58
I've compiled Python 3.3.6 using the same options (./configure --prefix=/usr/local --enable-shared) and build system and that passes almost all the tests (test_uuid fails for an ignorable reason).

Specifically test_hash passes fully:

$ LD_LIBRARY_PATH=/usr/local/src/Python-3.3.6 ./python -m test -v test_hash
== CPython 3.3.6 (default, Mar 26 2015, 15:35:36) [GCC 4.6.2]
==   Solaris-2.11-sun4v-sparc-32bit-ELF big-endian
==   /usr/local/src/Python-3.3.6/build/test_python_4539
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1)
[1/1] test_hash
test_empty_string (test.test_hash.BytesHashRandomizationTests) ... ok
test_fixed_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_null_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.DatetimeDateTests) ... ok
test_randomized_hash (test.test_hash.DatetimeDatetimeTests) ... ok
test_randomized_hash (test.test_hash.DatetimeTimeTests) ... ok
test_hashes (test.test_hash.HashBuiltinsTestCase) ... ok
test_coerced_floats (test.test_hash.HashEqualityTestCase) ... ok
test_coerced_integers (test.test_hash.HashEqualityTestCase) ... ok
test_numeric_literals (test.test_hash.HashEqualityTestCase) ... ok
test_unaligned_buffers (test.test_hash.HashEqualityTestCase) ... ok
test_default_hash (test.test_hash.HashInheritanceTestCase) ... ok
test_error_hash (test.test_hash.HashInheritanceTestCase) ... ok
test_fixed_hash (test.test_hash.HashInheritanceTestCase) ... ok
test_hashable (test.test_hash.HashInheritanceTestCase) ... ok
test_not_hashable (test.test_hash.HashInheritanceTestCase) ... ok
test_empty_string (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_fixed_hash (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_null_hash (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_empty_string (test.test_hash.StrHashRandomizationTests) ... ok
test_fixed_hash (test.test_hash.StrHashRandomizationTests) ... ok
test_null_hash (test.test_hash.StrHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.StrHashRandomizationTests) ... ok

----------------------------------------------------------------------
Ran 25 tests in 1.356s

OK
1 test OK.


So any ideas what I should look for? Or perhaps it would be helpful if I posted config.log, etc.
msg239352 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-26 21:31
Yes, 3.3 uses less efficient implementation.

Try to compile Python with gcc option -mno-unaligned-access.
msg239354 - (view) Author: Peter (petriborg) Date: 2015-03-26 21:57
That's not a valid option on SPARC, (see https://gcc.gnu.org/onlinedocs/gcc/SPARC-Options.html ) the flag is only available on ARM it seems.
msg239360 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2015-03-26 23:04
I'm puzzled about the segfault. I'm pretty sure that I've compiled and tested the code on big endian machines as well as a SPARC machines that doesn't allow unaligned memory access. The FNV code copies the blocks to an aligned buffer to prevent exactly this bug. Serhiy spent days to optimize the code as humanly possible.

Please recompile Python again. This time create a debug build (--with-pydebug) and use the default hash function FNV for your platform. The debug build helps to pin point the bug.
msg239384 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-27 09:59
What if add double field in the block union? (And may be compile with -munaligned-doubles)


diff -r a417d89fbc38 Python/pyhash.c
--- a/Python/pyhash.c   Thu Mar 26 09:37:23 2015 +0100
+++ b/Python/pyhash.c   Fri Mar 27 11:52:25 2015 +0200
@@ -247,6 +247,7 @@ fnv(const void *src, Py_ssize_t len)
     union {
         Py_uhash_t value;
         unsigned char bytes[SIZEOF_PY_UHASH_T];
+        double double_value;
     } block;
 
 #ifdef Py_DEBUG


What if add the __aligned__ attribute?

diff -r a417d89fbc38 Python/pyhash.c
--- a/Python/pyhash.c   Thu Mar 26 09:37:23 2015 +0100
+++ b/Python/pyhash.c   Fri Mar 27 11:58:30 2015 +0200
@@ -247,7 +247,7 @@ fnv(const void *src, Py_ssize_t len)
     union {
         Py_uhash_t value;
         unsigned char bytes[SIZEOF_PY_UHASH_T];
-    } block;
+    } block __attribute__ ((__aligned__(SIZEOF_PY_UHASH_T)));
 
 #ifdef Py_DEBUG
     assert(_Py_HashSecret_Initialized);
msg239385 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-27 10:04
Or may be better to use explicit alignment 8 if this is 32-bit platform.

diff -r a417d89fbc38 Python/pyhash.c
--- a/Python/pyhash.c   Thu Mar 26 09:37:23 2015 +0100
+++ b/Python/pyhash.c   Fri Mar 27 12:02:55 2015 +0200
@@ -247,7 +247,7 @@ fnv(const void *src, Py_ssize_t len)
     union {
         Py_uhash_t value;
         unsigned char bytes[SIZEOF_PY_UHASH_T];
-    } block;
+    } block __attribute__ ((__aligned__(8)));
 
 #ifdef Py_DEBUG
     assert(_Py_HashSecret_Initialized);
msg239386 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-27 10:40
The union trick is not portable, using memcpy() is safer. Compilers should be able to optimize it. Use Py_MEMCPY() for the stupid Visual Studio compiler unable to optimize memcpy() for small sizes.
msg239387 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-27 11:08
AFAIK it conforms the C standard and should be portable (but weird compiler bugs or misconfiguration can make it broken).

With Py_MEMCPY() the code is less optimal. Don't add regression on common platforms.
msg239393 - (view) Author: Peter (petriborg) Date: 2015-03-27 12:53
OK I recompiled with "./configure --prefix=/usr/local --enable-shared --with-pydebug" and reran the test, unfortunately...

$ LD_LIBRARY_PATH=/usr/local/src/Python-3.4.3 ./python -m test  test_hash
[1/1] test_hash
1 test OK.


I then applied the patch in msg239385, this resulted in the same crash, I couldn't see any difference in gdb.

I then tried each of the patches in msg239384, with -munaligned-doubles and with -mnounaligned-doubles, again no difference.

I did notice that two tests failed in test_hash prior to the core dump, so here are the outputs from those in case that helps.

<3.4.3 ./python -m test  -v test_hash.DatetimeDatetimeTests test_hash.DatetimeTimeTests
== CPython 3.4.3 (default, Mar 27 2015, 08:45:04) [GCC 4.6.2]
==   Solaris-2.11-sun4v-sparc-32bit-ELF big-endian
==   hash algorithm: fnv 32bit
==   /usr/local/src/Python-3.4.3/build/test_python_10340
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/2] test_hash.DatetimeDatetimeTests
test test_hash.DatetimeDatetimeTests crashed -- Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 2218, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/src/Python-3.4.3/Lib/test/regrtest.py", line 1271, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/usr/local/src/Python-3.4.3/Lib/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2221, in _find_and_load_unlocked
ImportError: No module named 'test.test_hash.DatetimeDatetimeTests'; 'test.test_hash' is not a package

[2/2/1] test_hash.DatetimeTimeTests
test test_hash.DatetimeTimeTests crashed -- Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 2218, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/src/Python-3.4.3/Lib/test/regrtest.py", line 1271, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/usr/local/src/Python-3.4.3/Lib/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2221, in _find_and_load_unlocked
ImportError: No module named 'test.test_hash.DatetimeTimeTests'; 'test.test_hash' is not a package

2 tests failed:
    test_hash.DatetimeDatetimeTests test_hash.DatetimeTimeTests



Thanks guys!
msg239397 - (view) Author: Peter (petriborg) Date: 2015-03-27 13:05
Sorry I copied the wrong term buffer :-)
This is the output after I commented out the HashEqualityTestCase class which causes the core dump.

LD_LIBRARY_PATH=/usr/local/src/Python-3.4.3 ./python -m test -v test_hash
== CPython 3.4.3 (default, Mar 27 2015, 08:45:04) [GCC 4.6.2]
==   Solaris-2.11-sun4v-sparc-32bit-ELF big-endian
==   hash algorithm: fnv 32bit
==   /usr/local/src/Python-3.4.3/build/test_python_10730
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_
site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_hash
test_empty_string (test.test_hash.BytesHashRandomizationTests) ... ok
test_fixed_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_long_fixed_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_null_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.BytesHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.DatetimeDateTests) ... ok
test_randomized_hash (test.test_hash.DatetimeDatetimeTests) ... FAIL
test_randomized_hash (test.test_hash.DatetimeTimeTests) ... FAIL
test_hashes (test.test_hash.HashBuiltinsTestCase) ... ok
test_hash_distribution (test.test_hash.HashDistributionTestCase) ... ok
test_default_hash (test.test_hash.HashInheritanceTestCase) ... ok
test_error_hash (test.test_hash.HashInheritanceTestCase) ... ok
test_fixed_hash (test.test_hash.HashInheritanceTestCase) ... ok
test_hashable (test.test_hash.HashInheritanceTestCase) ... ok
test_not_hashable (test.test_hash.HashInheritanceTestCase) ... ok
test_empty_string (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_fixed_hash (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_long_fixed_hash (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_null_hash (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.MemoryviewHashRandomizationTests) ... ok
test_empty_string (test.test_hash.StrHashRandomizationTests) ... ok
test_fixed_hash (test.test_hash.StrHashRandomizationTests) ... ok
test_long_fixed_hash (test.test_hash.StrHashRandomizationTests) ... ok
test_null_hash (test.test_hash.StrHashRandomizationTests) ... ok
test_randomized_hash (test.test_hash.StrHashRandomizationTests) ... ok
test_ucs2_string (test.test_hash.StrHashRandomizationTests) ... ok

======================================================================
FAIL: test_randomized_hash (test.test_hash.DatetimeDatetimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/src/Python-3.4.3/Lib/test/test_hash.py", line 193, in test_randomized_hash
    run1 = self.get_hash(self.repr_, seed='random')
  File "/usr/local/src/Python-3.4.3/Lib/test/test_hash.py", line 187, in get_hash
    **env)
  File "/usr/local/src/Python-3.4.3/Lib/test/script_helper.py", line 106, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/usr/local/src/Python-3.4.3/Lib/test/script_helper.py", line 92, in _assert_python
    err.decode('ascii', 'ignore')))
AssertionError: Process return code is -10, command line was: ['/usr/local/src/Python-3.4.3/python', '-X', 'faulthandle
r', '-c', 'import datetime; print(hash(datetime.datetime(1, 2, 3, 4, 5, 6, 7)))'], stderr follows:
Fatal Python error: Bus error

Current thread 0x00000001 (most recent call first):
  File "<string>", line 1 in <module>

======================================================================
FAIL: test_randomized_hash (test.test_hash.DatetimeTimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/src/Python-3.4.3/Lib/test/test_hash.py", line 193, in test_randomized_hash
    run1 = self.get_hash(self.repr_, seed='random')
  File "/usr/local/src/Python-3.4.3/Lib/test/test_hash.py", line 187, in get_hash
    **env)
  File "/usr/local/src/Python-3.4.3/Lib/test/script_helper.py", line 106, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/usr/local/src/Python-3.4.3/Lib/test/script_helper.py", line 92, in _assert_python
    err.decode('ascii', 'ignore')))
AssertionError: Process return code is -10, command line was: ['/usr/local/src/Python-3.4.3/python', '-X', 'faulthandle
r', '-c', 'import datetime; print(hash(datetime.time(0, 0)))'], stderr follows:
Fatal Python error: Bus error

Current thread 0x00000001 (most recent call first):
  File "<string>", line 1 in <module>

----------------------------------------------------------------------
Ran 26 tests in 3.429s

FAILED (failures=2)
test test_hash failed
msg239399 - (view) Author: Peter (petriborg) Date: 2015-03-27 13:13
Hi haypo,

I just realized you had created a patch too, the fnv_memcpy.patch worked!

$ LD_LIBRARY_PATH=/usr/local/src/Python-3.4.3 ./python -m test test_hash
[1/1] test_hash
1 test OK.

Running the full regression test now, but I bet everything passes.
msg239917 - (view) Author: Peter (petriborg) Date: 2015-04-02 14:24
So this morning I got around to rebuilding the tool chain using GCC 4.9.2 and I'm happy to report that this problem goes away (no patch required)! So it must be some sort of problem with the 4.6 GCC.

I've still got the old tool chain around, and I'm happy to further patch / test if anyone at Python wants me to.

$ gcc --verbose
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/sparc-sun-solaris2.11/4.9.2/lto-wrapper
Target: sparc-sun-solaris2.11
Configured with: ../gcc-4.9.2/configure --prefix=/usr/local --enable-languages=c,c++ --disable-nls --with-gnu-as --with-gnu-ld --target=sparc-sun-solaris2.11
Thread model: posix
gcc version 4.9.2 (GCC)
msg239930 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-02 16:03
Just interesting, what is the result of str(memoryview(b'abcdefghijklmnopqrstuvwxyz')[1:], 'ascii') with old and new toolchains.
msg240033 - (view) Author: Peter (petriborg) Date: 2015-04-03 20:25
Test 1
Python 3.4.3 built by GCC 4.9.2 is:
>>> str(memoryview(b'abcdefghijklmnopqrstuvwxyz')[1:], 'ascii')
'bcdefghijklmnopqrstuvwxyz'


Test 2
Python 3.4.3 built by GCC 4.6.2 is (no patches applied)
This build will core dump if I run -m test test_hash.
>>> str(memoryview(b'abcdefghijklmnopqrstuvwxyz')[1:], 'ascii')
'bcdefghijklmnopqrstuvwxyz'
msg395954 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-16 19:12
Can this be close (as third party?) or is there anything left to do?
msg395958 - (view) Author: Peter (petriborg) Date: 2021-06-16 20:30
We've migrated our python process off Solaris.
History
Date User Action Args
2021-06-16 20:30:58petriborgsetstatus: open -> closed

messages: + msg395958
stage: resolved
2021-06-16 19:12:53iritkatrielsetnosy: + iritkatriel
messages: + msg395954
2015-04-03 20:25:11petriborgsetmessages: + msg240033
2015-04-02 16:03:41serhiy.storchakasetmessages: + msg239930
2015-04-02 14:24:30petriborgsetmessages: + msg239917
2015-03-27 13:13:57petriborgsetmessages: + msg239399
2015-03-27 13:05:46petriborgsetmessages: + msg239397
2015-03-27 12:53:43petriborgsetmessages: + msg239393
2015-03-27 11:08:32serhiy.storchakasetmessages: + msg239387
2015-03-27 10:40:36vstinnersetfiles: + fnv_memcpy.patch
keywords: + patch
messages: + msg239386
2015-03-27 10:04:01serhiy.storchakasetmessages: + msg239385
2015-03-27 09:59:26serhiy.storchakasetmessages: + msg239384
2015-03-26 23:04:09christian.heimessetmessages: + msg239360
2015-03-26 21:57:38petriborgsetmessages: + msg239354
2015-03-26 21:33:04vstinnersetnosy: + vstinner
2015-03-26 21:31:00serhiy.storchakasetnosy: + mark.dickinson
messages: + msg239352
2015-03-26 19:58:33petriborgsetmessages: + msg239351
2015-03-26 18:18:18serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg239348
2015-03-26 17:57:05petriborgsetmessages: + msg239347
2015-03-26 13:50:30serhiy.storchakasetnosy: + jcea, christian.heimes

versions: + Python 3.5
2015-03-26 13:47:27petriborgcreate