This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Memory fault on SGI
Type: Stage:
Components: XML Versions: Python 2.4
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: fdrake Nosy List: bernhard, bos, fdrake, georg.brandl, kerofin, loewis, mhertha, mwh, nnorwitz, sf-robot, sjuranic
Priority: normal Keywords:

Created on 2004-11-30 13:13 by mhertha, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (17)
msg23376 - (view) Author: Maik Hertha (mhertha) Date: 2004-11-30 13:13
I build the latest RC1 of python 2.4.
System SGI Fuel IP35, Irix 6.5.26m

cc -version
MIPSpro Compilers: Version

- make tests passes test_pyexpat without error
- running this code leads to a core dump

-- code ---
import xml.dom.minidom
doc = xml.dom.minidom.parseString(fstream)

<<< core dump >>>
--- runing python -v
import xml.dom.expatbuilder # precompiled from
import xml.parsers # directory
import xml.parsers # precompiled from
# /opt/python24c1/lib/python2.4/xml/parsers/expat.pyc
matches /opt/python24c1/lib/python2.4/xml/parsers/
import xml.parsers.expat # precompiled from
Memory fault(coredump)

- running this code from an interactive session leads
not to a coredump

I need some assistance howto provide further debug

msg23377 - (view) Author: Stephen Watson (kerofin) Date: 2004-12-10 16:46
Logged In: YES 

I also got a core dump importing pyexpat on Solaris (SPARC)
and somebody else reported it on a *BSD system.

It appears to be a problem with older versions of expat not
being tested for.  I used gdb to trace it down to line 1972
in pyexpat.c where it attempts to define the first constant
from 1.95.8.  I had expat 1.95.7.  After upgrading to 1.95.8
it worked fine, as did the *BSD system. 

I think Python needs to check the expat version, as it does
elsewhere in the file, before defining those constants. 

I am still puzzelled over how it managed to compile
pyexpat.c in the first place...
msg23378 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-12-10 16:52
Logged In: YES 

Uh, Python includes its own copy of expat, and I really
thought we were supposed to prefer our own version over
anything found on the system...
msg23379 - (view) Author: Stephen Watson (kerofin) Date: 2004-12-10 17:01
Logged In: YES 

Maybe it compiled against its own copy of expat, but pulled
in the system's copy when run?  
msg23380 - (view) Author: Stephen Watson (kerofin) Date: 2004-12-13 09:29
Logged In: YES 

I've looked at the program that was dumping core and the
sequence is this:

1) Program imports pygtk, which links in the GTK libraries
2) Program loads SVG image which links in which
in turn links in /usr/local/lib/
3) Program imports pyexpat.
4) pyexpat calls XML_ErrorString, but as has already
linked in XML_ErrorString from /usr/local/lib/ it
calls that version, not the one in  
5) pyexpat looks up an error defined by the later version of
expat it is expecting and gets a NULL pointer from the
earlier version it has.  It attempts to use it without
checking (strlen) and dumps core.
msg23381 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-12-13 13:01
Logged In: YES 

1) Good sleuthing.
2) Argh!
3) What happens if you import pyexpat before pygtk?
msg23382 - (view) Author: Stephen Watson (kerofin) Date: 2004-12-13 13:46
Logged In: YES 

pyexpat initializes using its internal copy of expat. is still mapped in (after pyexpat has
initilized), but gdb finds the python copy of
XML_ErrorString and other functions.

I suspect that this will be a problem if the system version
of expat is newer than Python's.
msg23383 - (view) Author: Bryan O'Sullivan (bos) Date: 2005-02-01 07:53
Logged In: YES 

I've been bitten by the same bug. In my case, the problem was with Qt, not GTK.

It's not actually necessary to check the expat version; just see if XML_ErrorString actually returns anything.

Here's the patch I use for this problem:

--- python/Modules/pyexpat.c        2005-01-06 16:26:13 -08:00
+++ python/Modules/pyexpat.c  2005-01-31 23:46:36 -08:00
@@ -1936,9 +1936,12 @@

-#define MYCONST(name) \
-    PyModule_AddStringConstant(errors_module, #name, \
-                               (char*)XML_ErrorString(name))
+#define MYCONST(name)                           \
+    { \
+        char *_v = (char*)XML_ErrorString(name); \
+        if (_v) \
+            PyModule_AddStringConstant(errors_module, #name, _v); \
+    }

msg23384 - (view) Author: Bryan O'Sullivan (bos) Date: 2005-02-02 06:09
Logged In: YES 

By the way, this is not an SGI-specific bug. This will bite any Unix system with a modern sysv-style dynamic linker. The priority of this bug should be higher, IMO.
msg23385 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-02-02 10:35
Logged In: YES 

It's a nasty one, I'll give you that.

Fred, what do you think of bos's patch?  It solves the immediate issue, 
but I'm not sure it's a complete fix.

It seems to me that it would be better to resolve the expat symbols at 
builf time, but I don't know how to arrange for that.
msg23386 - (view) Author: Bryan O'Sullivan (bos) Date: 2005-02-04 23:16
Logged In: YES 

With the GNU linker, you can pass in -Bstatic to force it to resolve the symbols in the local shared library, instead of globally. This also works on Irix. I don't know about other Unixes.
msg23387 - (view) Author: Bernhard Herzog (bernhard) Date: 2005-03-29 18:11
Logged In: YES 

I ran into this problem as well on a debian GNU/Linux system
on x86 hardware.  It occurs both with pygtk 2.4 and wxPython
2.5 both built against gtk 2.4.

bos' patch at least solves the immediate problem of the
segmentation fault.  It may be a good idea to print a
warning message if some of the error codes cannot be
translated to strings, though.
msg23388 - (view) Author: Steve Juranich (sjuranic) Date: 2005-07-19 16:22
Logged In: YES 

FWIW, this same problem crops up when using Python & VTK
together (which also ships with its own version of expat). 
bos's workaround will make things work, though.
msg23389 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-09-30 06:01
Logged In: YES 

The problem seems to be related (same?) as bug #1295808. 
Does the patch there fix these problems?
msg23390 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2005-10-02 08:24
Logged In: YES 

I think the problem will occur whenever somebody loads with dlopen and RTLD_GLOBAL. IMO, this should be
avoided. I can't see in which of these different reports
this really is the case, but in those cases, this much looks
like a third-party bug.

If you import pyexpat first, it looks fine, since it will
resolve the symbols against its own copy, without exporting
them. Then, the later usage of libexpat cannot mistakenly go
into pyexpat because Python loaded this with RTLD_LOCAL.
However, it still may crash later when symbols are resolved
lazily, as more expat symbols might be needed when using
pyexpat, so different symbols might go to different libraries.

Linking pyexpat with -Bsymbolic (where available) should
also solve the problem.
msg23391 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-07-29 10:39
Logged In: YES 

This should be addressed by patch #1295808.
msg23392 - (view) Author: SourceForge Robot (sf-robot) Date: 2006-08-13 02:20
Logged In: YES 

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).
Date User Action Args
2022-04-11 14:56:08adminsetgithub: 41250
2004-11-30 13:13:31mherthacreate