Created on 2007-09-01 22:57 by sanders_muc, last changed 2007-12-04 15:39 by sanders_muc. This issue is now closed.
|findtest.c||sanders_muc, 2007-09-02 09:31|
|findtest.c||sanders_muc, 2007-09-02 10:47||Version sent to Intel|
|msg55567 - (view)||Author: Simon Anders (sanders_muc)||Date: 2007-09-01 22:59|
I have just encountered a strange bug affecting Python 2.5.1 on an x86_64 Linux, but only when compiled with the Intel C Compiler (ICC) 10.0, not a GCC-compiled Python. On my Intel-compiled one, which otherwise seems to work fine, ''.find() works incorrectly. I have narrowed down the issue to the simple test case "foo2/**bar**/".find ("/**bar**/") Observe: On a GCC-compiled Python 2.5.1, the command works as expected by returning 4: [c705213@hc-ma tmp]$ /usr/site/hc-2.6/python/gnu/2.5.1/bin/python2.5 Python 2.5.1 (r251:54863, Aug 30 2007, 16:21:23) [GCC 3.4.6 20060404 (Red Hat 3.4.6-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print "foo2/**bar**/".find ("/**bar**/") 4 On my Python 2.5.1 installation which was compiled from source with the Intel C Compiler (ICC) for Linux, version 10.0, '-1' is returned: [c705213@hc-ma tmp]$ /usr/site/hc-2.6/python/intel/2.5.1/bin/python2.5 Python 2.5.1 (r251:54863, Aug 30 2007, 16:20:06) [GCC Intel(R) C++ gcc 3.4 mode] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print "foo2/**bar**/".find ("/**bar**/") -1 What could have possibly gone wrong here? Admittedly, this smacks more of a bug in icc than in Python, but I report it here, as I feel at loss of what else to do with it. Obvious first question: Does anyone else out here have an ICC-compiled Python handy to check whether the bug reproduces elsewhere? Any idea what kind of oddity I have stumbled over here? Obviously, it could simply be that something went wrong when compiling Python from source with ICC, but it should not happen that the interpreter nebertheless starts up and fails only silently. Additional information: - I have stumbled over the problem when trying to install Numpy 220.127.116.11, as the built failed at the point where a script 'conv_template.py', which is part of NumPy's installtion system, is started to do some pattern replacements in a file called 'scalartypes.inc.src'. My test case is reduced from this script. - The system is the master node of a compute cluster with AMD Opteron CPUs. The cluster is not involved, all was done on the master node. The box runs RedHat Enterprise Linux 4.0 Advanced Server. It replies to 'uname -a' with: Linux hc-ma.uibk.ac.at 2.6.9-42.0.10.ELsmp #1 SMP Fri Feb 16 17:13:42 EST 2007 x86_64 x86_64 x86_64 GNU/Linux - The dynamic dependencies of the GCC-compiled and the ICC-compiled Python binaries are: [c705213@hc-ma tmp]$ ldd /usr/site/hc-2.6/python/gnu/2.5.1/bin/python2.5 libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000003702900000) libdl.so.2 => /lib64/libdl.so.2 (0x0000003701d00000) libutil.so.1 => /lib64/libutil.so.1 (0x0000003703900000) libm.so.6 => /lib64/tls/libm.so.6 (0x0000003701b00000) libc.so.6 => /lib64/tls/libc.so.6 (0x0000003701800000) /lib64/ld-linux-x86-64.so.2 (0x0000003701600000) [c705213@hc-ma tmp]$ ldd /usr/site/hc-2.6/python/intel/2.5.1/bin/python2.5 libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000003702900000) libdl.so.2 => /lib64/libdl.so.2 (0x0000003701d00000) libutil.so.1 => /lib64/libutil.so.1 (0x0000003703900000) libimf.so => /usr/site/hc-2.6/intel/10.0/cc/lib/libimf.so (0x0000002a95579000) libm.so.6 => /lib64/tls/libm.so.6 (0x0000003701b00000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003705800000) libc.so.6 => /lib64/tls/libc.so.6 (0x0000003701800000) /lib64/ld-linux-x86-64.so.2 (0x0000003701600000) - The precise revision of Python is "Python 2.5.1 (r251:54863)". - The test case ceases to show failure if the string is only slightly altered, e.g. if the word 'foo', the word 'bar' or the one of the asterisks or one of the slashes is cut out in both search and target string.
|msg55577 - (view)||Author: Martin v. Löwis (loewis) *||Date: 2007-09-02 07:08|
It definitely sounds like a compiler bug. Unless you can provide further details to the specific error in the C code of Python, it's likely that we can do little about it. If you want to analyze this further, here is a number of things you can try: - compile Python at various optimization levels. A compiler bug often manifests itself only at a specific set of optimization flags. - try tracing this invocation of .find() in a debugger. Doing so at a lower optimization level is easier, since the compiler may have inlined the various functions that form .find() under optimization. - if the debugger does not allow to pinpoint the erroneous function, add printf statements. Most of the code to study is in Objects/stringobject.c and Objects/stringlib/find.h.
|msg55579 - (view)||Author: Simon Anders (sanders_muc)||Date: 2007-09-02 09:31|
Martin, you are right: is is related to compiler optimization. I have boiled it down to a call of stringlib_find (defined in Python-2.5.1/Objects/stringlib/find.h) and this runs fine with 'icc -O2' but incorrectly for 'icc -O3'. (The test code is attached.) So, it seems that the lesson is simply once again: Do not use '-O3' with Intel's C compiler. (At least, for me, it is not the first time that this caused trouble.) On the other hand, Python's ./configure script is quite clear in its preference of GCC, anyway: It more or less ignores with option '--without-gcc' and uses the content of the CC environment variable only very occasionally.
|msg55580 - (view)||Author: Martin v. Löwis (loewis) *||Date: 2007-09-02 09:58|
If you are curious, we could now try to find out what precisely goes wrong. The procedure would be this * after each step, check whether the problem still occurs a) resolve the includes manually, then strip everything that isn't needed. This could start with fastsearch.h and find.h; then remove everything that refers to Python.h (i.e. replace Py_ssize_t with ssize_t, Py_LOCAL_INLINE with static inline, and so on), then remove Python.h b) try simplifying the code, e.g. replace str_len and sub_len with their (constant) values, drop the sub_len == 0 block, and so on. c) when further simplification is not possible (while keeping the actual error), start looking at the assembler code. Alternatively, sent this or some further-simplified version to Intel (assuming they have some kind of bug-reporting channel for icc). With my compiler-vendor's hat on, I'd like to get a test case for bad code generation that comes as a single file, with no includes; for gcc, the request is to submit preprocessor output. Assuming this is too much effort, I'll close this as "won't fix - third party".
|msg55582 - (view)||Author: Simon Anders (sanders_muc)||Date: 2007-09-02 10:47|
Martin: I've boiled down the test case a bit more and removed all Python-specific types and macros, so that it can now be compiled stand-alone. (Updated test case 'findtest.c' attached.) I didn't feel like diving into the code much deeper, and so I have sent it to Intel Premier Support as Issue #448807. Let's see if they bother to investigate it further.
|msg58188 - (view)||Author: Simon Anders (sanders_muc)||Date: 2007-12-04 15:39|
Update to the story: After I submitted the bug report to Intel, they investigated and quickly confirmed it to be a compiler bug, whcih they then managed to fix. I have just got an e-mail from Intel that the newest available version of ICC, namely version l_cc_c_10.1.008, contains the fix. In principle the problem should vanish now, but I have not found the time to verify that.
|2007-12-04 15:39:25||sanders_muc||set||messages: + msg58188|
messages: + msg55582
|2007-09-02 09:58:13||loewis||set||status: open -> closed|
resolution: wont fix
messages: + msg55580
versions: + 3rd party, - Python 2.5
messages: + msg55579
messages: + msg55577
messages: + msg55567