Title: expat symbols should be namespaced in pyexpat again
Type: crash Stage: resolved
Components: Extension Modules Versions: Python 3.3, Python 3.4
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Lukas.Vacek, Peter.Kruse, athompson, christian.heimes, fdrake, gregory.p.smith, python-dev
Priority: critical Keywords: patch

Created on 2013-10-07 13:04 by athompson, last changed 2014-05-15 08:01 by Peter.Kruse. This issue is now closed.

File name Uploaded Description Edit
fix_expat_names.patch Lukas.Vacek, 2014-02-04 14:40
with-pyexpatns.txt Peter.Kruse, 2014-03-19 09:39
without-pyexpatns.txt Peter.Kruse, 2014-03-19 09:39
Messages (7)
msg199143 - (view) Author: AThompson (athompson) Date: 2013-10-07 13:04
The issue with confliciting expat versions has poped up again in python 3.

It looks like the fix for this was removed when upgrading to expat 2.1 in      on Sat, 14 Jul 2012 14:12:35 -0700 Gregory P. Smith
--- a/Modules/expat/expat_external.h
/* Namespace external symbols to allow multiple libexpat version to
-   co-exist. */
-#include "pyexpatns.h"

Same backtrace as before.
#0  0x0000003507481321 in __strlen_sse2 () from /lib64/
#1  0x00007facace1b0d0 in PyUnicode_FromString (u=0x0) at Objects/unicodeobject.c:1824
#2  0x00007facaceefddc in PyModule_AddStringConstant (m=0x7fac9c7cf678,
    name=0x7fac9c5ab2e3 "XML_ERROR_UNBOUND_PREFIX", value=0x0) at Python/modsupport.c:554
#3  0x00007fac9c582f31 in PyInit_pyexpat () at /home/hex/Downloads/Python-3.Modules/pyexpat.c:1870
#4  0x00007facacee8f89 in _PyImport_LoadDynamicModule (name=0x7fac9c7db450, path=0x7fac9cdf9928, fp=0x0)
msg210234 - (view) Author: Lukas Vacek (Lukas.Vacek) Date: 2014-02-04 14:40
attaching patch
msg210240 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-04 15:12
New changeset c242a8f30806 by Benjamin Peterson in branch '3.3':
restore namespacing of pyexpat symbols (closes #19186)

New changeset a2d877fb53f6 by Benjamin Peterson in branch 'default':
merge 3.3 (#19186)
msg214069 - (view) Author: Peter Kruse (Peter.Kruse) Date: 2014-03-19 09:39

it seems that the solution to this issue causes the failure to 
compile the pyexpat extension in my case.  If I do not include
pyexpatns.h in expat_external.h then the compile succeeeds.
I will attach the output for both cases.  There was no problem
with 3.4.0b3.

My environment:

Python 3.4.0
Red Hat Linux 6.4
GCC 4.8.2

it makes no difference if I use --with-system-expat or without.
msg218169 - (view) Author: Lukas Vacek (Lukas.Vacek) Date: 2014-05-09 15:02
Hi Peter,

Thanks for taking the time to report your issue. I just tried on fresh up-to-date CentOS 6 and I could not reproduce the issue.

However, I think I figured out what went wrong. I can see you are supplying custom -I and -L paths (-I/apps/prod/releases/3.0/include ...) I suspect that when compiling pyexpat.c (in Modules/) the header file expat.h is picked up from this location before the bundled expat.h (in Modules/expat/) is found. But when compiling xmlparse.c (in Modules/expat/) the bundled header file expat.h *is* used this time. So we end up with our bundled expat compiled using namespacing and so defining symbols like PyExpat_XML_SetCommentHandler while pyexpat.c is looking for XML_SetCommentHandler because it is compiled *not* using our bundled expat.h.

I am not sure yet why your custom "-I" comes before "-I/ae/data/soft/opensource/build/python3/master/Python-3.4.0/Modules/expat" as it certainly should not. 

So far I tried building with CPPFLAGS and CFLAGS and in both cases my custom paths come after "-I.../Modules/expat" as it should.

Can you please provide your ./configure and make command lines and relevant environment variables?

msg218172 - (view) Author: Lukas Vacek (Lukas.Vacek) Date: 2014-05-09 15:55
Actually CFLAGS do indeed come before any include directories.

I can reproduce your problem:
mkdir /tmp/extra_include
cp /usr/include/expat.h /usr/include/expat_external.h /tmp/extra_include/
make CFLAGS="-I/tmp/extra_include"

You should use CPPFLAGS for extra include directories. Not an issue in CPython.
msg218598 - (view) Author: Peter Kruse (Peter.Kruse) Date: 2014-05-15 08:01
Fascinating, you are right, very good, thanks for your time and looking into this.

Date User Action Args
2014-05-15 08:01:28Peter.Krusesetmessages: + msg218598
2014-05-09 15:55:16Lukas.Vaceksetmessages: + msg218172
2014-05-09 15:02:31Lukas.Vaceksetmessages: + msg218169
2014-03-19 09:39:18Peter.Krusesetfiles: + without-pyexpatns.txt
2014-03-19 09:39:01Peter.Krusesetfiles: + with-pyexpatns.txt
nosy: + Peter.Kruse
messages: + msg214069

2014-02-04 15:12:26python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg210240

resolution: fixed
stage: needs patch -> resolved
2014-02-04 14:40:55Lukas.Vaceksetfiles: + fix_expat_names.patch

nosy: + Lukas.Vacek
messages: + msg210234

keywords: + patch
2013-10-08 08:45:59pitrousetpriority: normal -> critical
2013-10-07 13:45:58pitrousetnosy: + gregory.p.smith
2013-10-07 13:24:39christian.heimessetnosy: + fdrake, christian.heimes

stage: needs patch
2013-10-07 13:04:57athompsoncreate