classification
Title: NetBSD: segmentation fault in listextend during install
Type: crash Stage: resolved
Components: Installation Versions: Python 2.6
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: jon.buller, loewis, ned.deily, pitrou, rpointel, vstinner, xuhdev
Priority: normal Keywords:

Created on 2009-12-02 04:57 by jon.buller, last changed 2014-06-07 18:21 by ned.deily. This issue is now closed.

Messages (14)
msg95895 - (view) Author: Jon Buller (jon.buller) Date: 2009-12-02 04:57
On a NetBSD/sparc-current system building from the 2.6.4.tgz file...

Compiling /usr/pkg/lib/python2.6/test/test_bool.py ...
Compiling /usr/pkg/lib/python2.6/test/test_bsddb.py ...
Compiling /usr/pkg/lib/python2.6/test/test_bsddb185.py ...
Compiling /usr/pkg/lib/python2.6/test/test_bsddb3.py ...
Compiling /usr/pkg/lib/python2.6/test/test_buffer.py ...
Compiling /usr/pkg/lib/python2.6/test/test_bufio.py ...
Compiling /usr/pkg/lib/python2.6/test/test_builtin.py ...
[1]   Segmentation fault (core dumped) PYTHONPATH=/usr/...
*** Error code 139

Stop.


and GDB says...

Reading symbols from /usr/lib/libc.so.12...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/libc.so.12
Reading symbols from
/usr/pkgsrc/lang/python26/work/Python-2.6.4/build/lib.netbsd-5.99.15-sparc-2.6/unicodedata.so...
(no debugging symbols found)...done.
Loaded symbols for
/usr/pkgsrc/lang/python26/work/Python-2.6.4/build/lib.netbsd-5.99.15-sparc-2.6/unicodedata.so
Reading symbols from /usr/libexec/ld.elf_so...(no debugging symbols
found)...done.
Loaded symbols for /usr/libexec/ld.elf_so

Core was generated by `python'.
Program terminated with signal 11, Segmentation fault.
#0  0x200b7bac in listextend ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
(gdb) where
#0  0x200b7bac in listextend ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
#1  0x20087b00 in PySequence_List ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
#2  0x20129760 in assemble ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6..so.1.0
#3  0x2012e12c in compiler_function ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
#4  0x2012e288 in compiler_body ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
#5  0x2012e498 in compiler_class ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
#6  0x2012e288 in compiler_body ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
#7  0x2012e958 in PyAST_Compile ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0
#8  0x20142a48 in Py_CompileStringFlags ()
   from /usr/pkgsrc/lang/python26/work/Python-2.6.4/libpython2.6.so.1.0

...
msg100832 - (view) Author: Jon Buller (jon.buller) Date: 2010-03-11 03:31
Sorry to not be able to follow up for so long, but I was moving 
cross-country.

I was playing with the tests a bit and ended up trying this:

$ ./python -E -tt -d -v -W all ./Lib/test/regrtest.py -l -v -s test_builtin

Which resulted (after quite a bit of output) in this:

# /usr/pkgsrc/lang/python26/work/Python-2.6.4/Lib/tempfile.pyc matches 
/usr/pkgsrc/lang/python26/work/Python-2.6.4/Lib/tempfile.py
import tempfile # precompiled from 
/usr/pkgsrc/lang/python26/work/Python-2.6.4/Lib/tempfile.pyc
dlopen("/usr/pkgsrc/lang/python26/work/Python-2.6.4/build/lib.netbsd-5.99.24-sparc-2.6/fcntl.so", 
2);
import fcntl # dynamically loaded from 
/usr/pkgsrc/lang/python26/work/Python-2.6.4/build/lib.netbsd-5.99.24-sparc-2.6/fcntl.so
import thread # builtin
test_builtin
[1]   Segmentation fault      ./python -E -tt -d -v -W all 
./Lib/test/regrtest.py -l -v -s test_builtin

So, is this perhaps a bug either in Python's thread usage, or NetBSD's 
thread
library?  Is there an easy way to tell python to build without any threads,
even if configure finds the proper system include files and libraries?  Is
it as simple as "./configure --without-threads" when starting the build?
(I'm about to try that...)

Jon
msg100844 - (view) Author: Jon Buller (jon.buller) Date: 2010-03-11 12:26
"./configure --without-threads" didn't seem to have any effect...  :(
msg100893 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-03-11 23:15
Can you try to recompile Python in debug mode?
("make distclean && ./configure --with-pydebug").
msg100897 - (view) Author: Jon Buller (jon.buller) Date: 2010-03-12 00:16
This is bad.  The problem went away and the test completed without the 
segfault.  If you think the output would help track anything down, let 
me know and I'll
save and upload it somewhere.  (Or I could hand out an SSH account via 
IPv6 to the machine...  With some effort, I could probably reconfigure 
my network to get it
on one of my 2 IPv4 addresses.)
msg103283 - (view) Author: Jon Buller (jon.buller) Date: 2010-04-16 01:36
I found that python 2.4.5 will compile and install on this machine,
though I don't know if that helps anyone or not, or how to track this
down further.  (At least I can run mercurial on that machine again.)

On 03/11/10 18:16, Jon Buller wrote:
>
> Jon Buller<jon@bullers.net>  added the comment:
>
> This is bad.  The problem went away and the test completed without the
> segfault.  If you think the output would help track anything down, let
> me know and I'll
> save and upload it somewhere.  (Or I could hand out an SSH account via
> IPv6 to the machine...  With some effort, I could probably reconfigure
> my network to get it
> on one of my 2 IPv4 addresses.)
>
> ----------
>
> _______________________________________
> Python tracker<report@bugs.python.org>
> <http://bugs.python.org/issue7424>
> _______________________________________
msg127827 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-02-03 22:28
See also:

http://gnats.netbsd.org/42627
msg141599 - (view) Author: Remi Pointel (rpointel) * Date: 2011-08-03 05:16
Hi,
we have the same problem on OpenBSD (sparc).
You could find more information:
http://marc.info/?l=openbsd-ports&m=131219537505698
Thanks,
Remi.
msg141612 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-08-03 16:04
Just a guess: it might be a stack overflow. Could you try to reduce
Py_DEFAULT_RECURSION_LIMIT in Python/ceval.c and see if you get
a RuntimeError instead?

See also: issue #1212900
msg141629 - (view) Author: Remi Pointel (rpointel) * Date: 2011-08-04 14:14
I changed the value from 1000 to 200, and it crashed the same way, then I tested with 20, and it ran into a runtime error but in a different place:

c++ -pthread -L/home/ports/pobj/Python-2.7.1/Python-2.7.1 -Wl,--export-dynamic -o python  Modules/python.o  -lpython2.7 -pthread  -lutil   -lm  
/home/ports/pobj/Python-2.7.1/Python-2.7.1/libpython2.7.so.0.0: warning: tmpnam() possibly used unsafely; consider using mkstemp()
/home/ports/pobj/Python-2.7.1/Python-2.7.1/libpython2.7.so.0.0: warning: tempnam() possibly used unsafely; consider using mkstemp()
/home/ports/pobj/Python-2.7.1/Python-2.7.1/libpython2.7.so.0.0: warning: strcpy() is almost always misused, please use strlcpy()
/home/ports/pobj/Python-2.7.1/Python-2.7.1/libpython2.7.so.0.0: warning: sprintf() is often misused, please use snprintf()
/home/ports/pobj/Python-2.7.1/Python-2.7.1/libpython2.7.so.0.0: warning: strcat() is almost always misused, please use strlcat()
Traceback (most recent call last):
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/site.py", line 553, in <module>
    main()
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/site.py", line 535, in main
    known_paths = addusersitepackages(known_paths)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/site.py", line 268, in addusersitepackages
    user_site = getusersitepackages()
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/site.py", line 243, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/site.py", line 233, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sysconfig.py", line 535, in get_config_var
    return get_config_vars().get(name)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sysconfig.py", line 434, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sysconfig.py", line 282, in _init_posix
    _parse_makefile(makefile, vars)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sysconfig.py", line 195, in _parse_makefile
    _variable_rx = re.compile("([a-zA-Z][a-zA-Z0-9_]+)\s*=\s*(.*)")
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/re.py", line 190, in compile
    return _compile(pattern, flags)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/re.py", line 243, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sre_compile.py", line 500, in compile
    p = sre_parse.parse(p, flags)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sre_parse.py", line 673, in parse
    p = _parse_sub(source, pattern, 0)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sre_parse.py", line 308, in _parse_sub
    itemsappend(_parse(source, state))
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sre_parse.py", line 633, in _parse
    p = _parse_sub(source, state)
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sre_parse.py", line 308, in _parse_sub
    itemsappend(_parse(source, state))
  File "/home/ports/pobj/Python-2.7.1/Python-2.7.1/Lib/sre_parse.py", line 381, in _parse
    subpattern = SubPattern(state)
RuntimeError: maximum recursion depth exceeded while calling a Python object
*** Error code 1

Then I tried again with the value set to 100, and it ran into the original segfault again.

Thanks for your help.
msg141630 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-08-04 14:22
The problem with such issues is that:
- they are not reproduceable by someone of the core team
- they only seem to occur in non-debug mode
- they are probably tied to a specific compiler variant, either because of a compiler bug, or some aggressive optimization combined with some misled assumption in Python's code

To narrow it down, you could:
- try with other compiler versions
- try with other Python versions from python.org (for example the recent 3.2.1)
msg141633 - (view) Author: Remi Pointel (rpointel) * Date: 2011-08-04 21:10
Thanks Antoine for your response.
Python 3.2.1 and Python 2.4.6 have been successfully installed (but Python 2.5.4 and Python 2.7.1 failed).
msg141644 - (view) Author: Jon Buller (jon.buller) Date: 2011-08-05 01:35
I recently upgraded my SparcStation20 to NetBSD-current and thought I 
would try installing python again.  I found that both 2.7.1 and whatever 
was the most current 2.5 pkgsrc version as of a few months ago built and 
installed just fine.  I'm sorry I can no longer help debug this, but the 
bug was very repeatable on 2.6 and 2.7 for several years and often 
happened with 2.5 as well. But I can no longer repeat it.  I also can't 
say if it is a GCC bug, a NetBSD kernel or libc bug, or a python bug. I 
was sort of expecting it to be a NetBSD bug since I don't remember ever 
having problems with x86, amd64, or sparc64 NetBSD systems, but I still 
believe it could be just about anywhere.

Jon
msg219957 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-06-07 18:21
See issue12673 msg219946.  Apparently this was caused by a long-standing BSD sparc bug.
History
Date User Action Args
2014-06-07 18:21:10ned.deilysetstatus: open -> closed

nosy: + ned.deily
messages: + msg219957

resolution: third party
stage: test needed -> resolved
2014-05-14 15:35:39vstinnersettitle: segmentation fault in listextend during install -> NetBSD: segmentation fault in listextend during install
2014-05-13 21:48:06skrahsetnosy: - skrah
2011-08-05 01:35:35jon.bullersetmessages: + msg141644
2011-08-04 21:10:06rpointelsetmessages: + msg141633
2011-08-04 14:22:18pitrousetmessages: + msg141630
2011-08-04 14:14:35rpointelsetmessages: + msg141629
2011-08-03 16:04:11skrahsetmessages: + msg141612
2011-08-03 05:16:11rpointelsetmessages: + msg141599
2011-08-01 12:47:48rpointelsetnosy: + rpointel
2011-08-01 12:45:21r.david.murraylinkissue12673 superseder
2011-02-03 22:28:57skrahsetnosy: + xuhdev, skrah, loewis, vstinner
messages: + msg127827
2011-02-03 22:28:50skrahlinkissue11106 superseder
2010-04-16 01:36:56jon.bullersetmessages: + msg103283
2010-03-12 00:16:23jon.bullersetmessages: + msg100897
2010-03-11 23:15:57pitrousetnosy: + pitrou
messages: + msg100893
2010-03-11 12:26:16jon.bullersetmessages: + msg100844
2010-03-11 03:31:50jon.bullersetmessages: + msg100832
2010-02-09 17:07:14brian.curtinsetpriority: normal
stage: test needed
2009-12-02 04:57:16jon.bullercreate