classification
Title: urllib2.urlopen() segfault using SSL on Solaris
Type: crash Stage: resolved
Components: Extension Modules Versions: Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, asdfasdfasdfasdfasdfasdfasdf, grobian, orsenthil, rtarnell
Priority: normal Keywords:

Created on 2009-11-11 13:42 by rtarnell, last changed 2011-03-04 13:32 by asdfasdfasdfasdfasdfasdfasdf. This issue is now closed.

Messages (11)
msg95139 - (view) Author: River Tarnell (rtarnell) Date: 2009-11-11 13:42
Using Python 2.6.2 or 2.6.4 on Solaris 10, opening an SSL (https) URL 
using urllib2.urlopen() crashes Python:

% python
Python 2.6.4 (r264:75706, Nov 11 2009, 13:23:45) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2; urllib2.urlopen('https://launchpad.net/~netbook-
remix-team/+archive/ppa')
zsh: segmentation fault (core dumped)  python
% pstack core
core 'core' of 14970:   python
 fe86663f AES_encrypt () + 173
%

Python was compiled using Sun C 5.10 SunOS_i386 2009/06/03 and OpenSSL 
0.9.7d (+ Sun security patches) using flags:

CFLAGS='-i -xO4 -xspace -xstrconst -xpentium -mr -xregs=no%frameptr'
LDFLAGS='-L/opt/ts/python/2.6/lib -L/usr/sfw/lib -R/opt/ts/python/2.6/
lib:/usr/sfw/lib -L/opt/ts/X11/lib -R/opt/ts/X11/lib -L/opt/ts/lib -R/
opt/ts/lib -L/opt/ts/python/2.6/lib'
CPPFLAGS='-I/opt/ts/python/2.6/include -I/usr/sfw/include -I/opt/ts/X11/
include -D__EXTENSIONS__ -I/opt/ts/include'

./configure --prefix=/opt/ts/python/2.6 --bindir=/opt/ts/python/2.6/bin 
--libdir=/opt/ts/python/2.6/lib --mandir=/opt/ts/python/26/share/man --
datadir=/opt/ts/python/2.6/share --includedir=/opt/ts/python/2.6/
include --infodir=/opt/ts/python/2.6/share/info --libexecdir=/opt/ts/
python/2.6/lib --sysconfdir=/etc/opt/ts --disable-nls --disable-static 
--with-threads --enable-ipv6 --prefix=/opt/ts/python/2.6 --enable-shared
msg112888 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-04 20:35
Can someone with a Solaris box try to reproduce this.
msg128545 - (view) Author: Fabian Groffen (grobian) Date: 2011-02-14 13:54
I can reproduce this on Solaris 10/Sparc64 only, using Python 2.7.1.  My core files aren't really useful though, nothing it can point to, everything is corrupt.
msg128547 - (view) Author: Fabian Groffen (grobian) Date: 2011-02-14 14:04
Perhaps I should have been a bit more clear.  Python 2.6 works fine for me on Solaris 10/Sparc64 (64-bits).  Python 2.7.1 also works on Solaris 10/Sparc (32-bits), but not on Sparc64 (64-bits).
msg128580 - (view) Author: Fabian Groffen (grobian) Date: 2011-02-15 09:29
Sorry for the noise.  My issue seems to be caused by a broken openssl.  It is a specific problem with the code for sparcv9 and the T1 CPU.  Python can't do much about this.
msg128581 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2011-02-15 09:42
Thanks for the update on this issue.
msg128588 - (view) Author: River Tarnell (rtarnell) Date: 2011-02-15 12:13
If no one else can reproduce this I suppose there's no point leaving it open, but it's still present for me using Python 2.7.1:

% bzip2 -dc Python-2.7.1.tar.bz | tar xvf -
% cd Python-2.7.1
% CC=cc CXX=CC CPPFLAGS='-I/usr/sfw/include' LDFLAGS='-L/usr/sfw/lib -R/usr/sfw/lib' ./configure --enable-ipv6 --prefix=$HOME/pytest
% gmake -j5
% gmake install
% $HOME/pytest/bin/python
Python 2.7.1 (r271:86832, Feb 15 2011, 12:06:37) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> urllib2.urlopen('https://wiki.toolserver.org')
zsh: segmentation fault (core dumped)  $HOME/pytest/bin/python
% pstack core
core 'core' of 12301:   /home/rriver/pytest/bin/python
 feb265bd AES_encrypt () + f1
% cat /etc/release
                    Oracle Solaris 10 9/10 s10x_u9wos_14a X86
msg128589 - (view) Author: River Tarnell (rtarnell) Date: 2011-02-15 12:15
Also, when compiled with OpenSSL 1.0.0c, the problem doesn't occur.
msg128590 - (view) Author: Fabian Groffen (grobian) Date: 2011-02-15 12:25
I recompiled openssl with noasm flag now for sparcv9 (like I do on 32-bits compiles), and the example runs fine for me on the T1, proving Python is not to blame here.

That said, if your problem only occurs with 0.9.8 isn't it then likely that that version has some bug which is fixed in 1.0.0 series?
My gdb backtraces pointed at code inside libcrypto (one of the ASM accelerators).

I am using GCC and a minimum of host-provided libs though.

% python2.7 -V
Python 2.7.1 (r271:86832, Feb 14 2011, 12:37:15) 
[GCC 4.3.2] on sunos5
% file `which python2.7`
/scratch/tmp/gentoo/sparcv9/usr/bin/python2.7: ELF 64-bit MSB executable, SPARC V9, total store ordering, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
% uname -a
SunOS build-t1000-sol10.cwi.nl 5.10 Generic_139555-08 sun4v sparc SUNW,SPARC-Enterprise-T1000 Solaris
msg128603 - (view) Author: River Tarnell (rtarnell) Date: 2011-02-15 16:06
Well, I can "prove" it's not OpenSSL's fault either, since "openssl s_client" works fine ;-)  (As do many other OpenSSL-using applications.)

In any case, since I'm not using SPARC, it's unlikely that a bug in the SPARC asm code is the cause of the problem.  Of course it could still be some other OpenSSL bug, or it could be that Python is using OpenSSL wrongly (and perhaps something changed between 0.9.7 and 1.0.0 that prevents the problem occuring).

We now use our own OpenSSL instead of Sun's, so this isn't a problem anymore, but it will affect anyone using Python on Solaris 10 with Sun's OpenSSL.
msg130052 - (view) Author: david (asdfasdfasdfasdfasdfasdfasdf) Date: 2011-03-04 13:32
I have also hit this bug. It is slightly interesting that urllib is able to connect to hosts that trigger a segfault under urllib2  without an issue...
History
Date User Action Args
2011-03-04 13:32:40asdfasdfasdfasdfasdfasdfasdfsetnosy: + asdfasdfasdfasdfasdfasdfasdf
messages: + msg130052
2011-02-15 16:06:07rtarnellsetnosy: orsenthil, rtarnell, grobian, BreamoreBoy
messages: + msg128603
2011-02-15 12:25:05grobiansetnosy: orsenthil, rtarnell, grobian, BreamoreBoy
messages: + msg128590
2011-02-15 12:15:05rtarnellsetnosy: orsenthil, rtarnell, grobian, BreamoreBoy
messages: + msg128589
2011-02-15 12:13:16rtarnellsetnosy: orsenthil, rtarnell, grobian, BreamoreBoy
messages: + msg128588
2011-02-15 09:42:49orsenthilsetstatus: open -> closed

nosy: + orsenthil
messages: + msg128581

resolution: not a bug
stage: resolved
2011-02-15 09:29:47grobiansetnosy: rtarnell, grobian, BreamoreBoy
messages: + msg128580
versions: - Python 2.7
2011-02-14 14:04:43grobiansetnosy: rtarnell, grobian, BreamoreBoy
messages: + msg128547
2011-02-14 13:54:57grobiansetnosy: + grobian

messages: + msg128545
versions: + Python 2.7
2010-08-04 20:35:19BreamoreBoysetnosy: + BreamoreBoy
messages: + msg112888
2009-11-11 13:42:50rtarnellcreate