Issue1759169
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2007-07-23 20:24 by zooko, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
pypatch.diff | zooko, 2007-07-23 20:24 |
Messages (23) | |||
---|---|---|---|
msg52938 - (view) | Author: Zooko O'Whielacronx (zooko) | Date: 2007-07-23 20:24 | |
I wanted to build a Python extension module written in C99, but Solaris's headers ("sys/feature_test.h") declare that you can't have an XPG 4.2 application using C99. Investigating led me eventually to Solaris hacker John Levon, who kindly pointed out to me that in the Sun-maintained port of Python, they use a perl script to strip out _XOPEN_SOURCE, _XOPEN_SOURCE_EXTENDED, and _POSIX_SOURCE from pyconfig.h: http://src.opensolaris.org/source/xref/jds/spec-files/trunk/SUNWPython.spec (See lines 123-125.) The attached patch accomplishes the same goal more cleanly by changing the contents of configure.in. With this patch, I can now build python trunk and release25-maint and also build a C99 extension module: http://cheeseshop.python.org/pypi/zfec This patch might also impact "[ 1306253 ] Python 2.4.2c1 fails to build on 64-bit Solaris 10": https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1306253&group_id=5470 I think perhaps this patch should supercede the patch mentioned in "[ 1116722 ] Solaris 10 fails to compile complexobject.c [FIX incl.]": https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1116722&group_id=5470 |
|||
msg52939 - (view) | Author: Georg Brandl (georg.brandl) * | Date: 2007-07-24 22:02 | |
Looks like a patch for you, Martin... |
|||
msg55588 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2007-09-02 19:14 | |
I don't understand the problem. What Solaris version are you using? Why do you want to remove _XOPEN_SOURCE? Solaris considers this as a request for XPG 4.2 only _XOPEN_SOURCE_EXTENDED is defined, which it shouldn't be on Solaris 10. |
|||
msg73709 - (view) | Author: Zooko O'Whielacronx (zooko) | Date: 2008-09-24 12:23 | |
Sorry I didn't get back to this ticket, MvL. Recently someone trying to build the Tahoe distributed filesystem on Solaris had a problem: http://allmydata.org/pipermail/tahoe-dev/2008-September/000789.html They had built Python 2.5 themselves from source. It turned out to be this issue: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6395191 Which is that g++ on Solaris fails when _XOPEN_SOURCE_EXTENDED is defined. This reminded me of this ticket, so here I am again. So, here is the port of Python to Solaris that is maintained by the Solaris engineers: http://src.opensolaris.org/source/xref/jds/spec-files/trunk/SUNWPython.spec It contains the following stanza: 146 # These #define's break the build with gcc and cause problems when 147 # building c99 compliant python modules 148 perl -pi -e 's/^(#define _POSIX_C_SOURCE.*)/\/* $1 *\//' pyconfig.h 149 perl -pi -e 's/^(#define _XOPEN_SOURCE.*)/\/* $1 *\//' pyconfig.h 150 perl -pi -e 's/^(#define _XOPEN_SOURCE_EXTENDED.*)/\/* $1 *\//' pyconfig.h This shows that the Python that comes with Solaris has, for a long time now, had these #define's stripped out of the pyconfig.h. Why? Well, I asked the Solaris engineers about it on IRC, and they were very nice and helpful, and they said, if I am remembering correctly, that Solaris doesn't use these #define's in the way that we Python people think. #defining _XOPEN_SOURCE_EXTENDED on Solaris is *not* understood by the Solaris runtime as a request for XPG4v2 (UNIX98) features to be supported, but more as a declaration that the code we are compiling was written for XPG4v2. So, for example, this disables C99, because after all C99 wasn't available in XPG4v2. It also apparently breaks g++, but that is by accident rather than design. In any case, the Solaris engineers assured me that rather than figuring out which unixy features are turned on by which #defines, we should instead #define __EXTENSIONS__ and then assume all the features we can eat. Whether this is true in general remains to be seen, but apparently _POSIX_C_SOURCE, _XOPEN_SOURCE, and _XOPEN_SOURCE_EXTENDED, at least, are proven to be unnecessary on Solaris by the fact that the version of Python that has been shipping with Solaris for who knows how long has them stripped out. Therefore, I propose to apply my patch so that we do not turn on _XOPEN_SOURCE but we do turn on __EXTENSIONS__, when building for Solaris. |
|||
msg80949 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2009-02-02 16:35 | |
If the Sun guy only commented on _XOPEN_SOURCE_EXTENDED, I'm not sure that this also applies to _XOPEN_SOURCE. For example, on my Solaris 9, I have (in signal.h) conditionally #pragma redefine_extname sigwait __posix_sigwait and in sys/socket.h #pragma redefine_extname connect __xnet_connect So these feature selection macros not only select what API is declared in the header files, but also what specific semantics some of the functions have (e.g. if the traditional BSD semantics deviates from the POSIX semantics). In cases where there are alternative implementations of some system calls, we *want* to select the POSIX definition. So I think we should continue to define _XOPEN_SOURCE. I don't mind also defining __EXTENSIONS__. |
|||
msg82201 - (view) | Author: John Levon (movement) | Date: 2009-02-16 01:22 | |
Just came across this bug. _XOPEN_SOURCE_EXTENDED isn't the only sticking point here: neither XPG3 nor XPG4 have C99 either: _XOPEN_SOURCE (at least with a value below 600) is definitely wrong. To quote the header: "346 * It is invalid to compile an XPG3, XPG4, XPG4v2, or XPG5 application 347 * using c99. The same is true for POSIX.1-1990, POSIX.2-1992, POSIX.1b, 348 * and POSIX.1c applications. Likewise, it is invalid to compile an XPG6 349 * or a POSIX.1-2001 application with anything other than a c99 or later 350 * compiler. Therefore, we force an error in both cases. " Martin, you are correct that these macros choose particular semantics in some cases. However, this does not change my point. It is a fact that specifying these macros mean what I describe, and that doing so causes significant problems in compiling Python extensions. We may wish it to be different, but it's not. As Solaris has only ever moved towards the POSIX spec, __EXTENSIONS__ is the right thing to always specify here: it enables everything "modern" that you need (*including* POSIX semantics - check that signal.h on Solaris 9 for __EXTENSIONS__), without making any claims about conformity of Python extensions (which it it is not pyconfig.h's place to say!). I hope that's a clearer explanation of why Python is doing the wrong thing both theoretically and practically for Solaris. |
|||
msg82203 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2009-02-16 02:06 | |
> As Solaris has only ever moved towards the POSIX spec, __EXTENSIONS__ is > the right thing to always specify here And indeed, Python does define __EXTENSIONS__. > I hope that's a clearer explanation of why Python is doing the wrong > thing both theoretically and practically for Solaris. No. You are arguing that Python should define __EXTENSIONS__ when it already does. This doesn't help to clarify. |
|||
msg82212 - (view) | Author: John Levon (movement) | Date: 2009-02-16 02:59 | |
Python should define *only* __EXTENSIONS__, not the others, as Zooko already mentioned, for the reasons he mentioned. I'm not sure how you got a different impression from what I or he said. |
|||
msg82216 - (view) | Author: Zooko O'Whielacronx (zooko) | Date: 2009-02-16 05:02 | |
Martin (sometimes called "MvL") mentioned some specific issues about e.g. #pragma redefine_extname sigwait __posix_sigwait I didn't understand exactly what MvL's concern was, and I don't know off the top of my head how Solaris sigwait behavior differs with _XOPEN_SOURCE, and I haven't taken the time to investigate yet, so I haven't replied. (Of course, last time I was in that state and I dropped the ball, this ticket stayed quiet for a full year...) Perhaps John Levon could inquire what semantics MvL is concerned about and let him know what the Solaris behavior is. I think it would be cool if the python executable that people use on Solaris were closer to upstream python. Currently, as I've mentioned, the Solaris folks maintain some kind of tool (I think it is a perl script) which searches and replaces "#define _XOPEN_SOURCE" to empty as well as applying a few patches. It would be great if the Solaris and Python folks would cooperate to close that gap. Speaking of which, if John Levon or other Solaris folk are reading -- where are those patches? This file mentions a bunch of patches, starting with: http://src.opensolaris.org/source/xref/jds/spec-files/trunk/SUNWPython.spec 28 Patch1: Python-01-solaris-lib-dirs.diff But I can't seem to find "Python-01-solaris-lib-dirs.diff". Thanks! --Zooko |
|||
msg82223 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2009-02-16 08:17 | |
> Python should define *only* __EXTENSIONS__, not the others, as Zooko > already mentioned, for the reasons he mentioned. I'm not sure how you > got a different impression from what I or he said. Ok, so we can ignore __EXTENSIONS__ for the discussion. Then you were arguing that defining _XOPEN_SOURCE is wrong if the value is less than 600. What about setting the value to 600 or larger? Is that also wrong, and if so, why? [Aside: I think the Solaris comment on using C99 with XOPEN applications is misguided. Compiling such applications with C99 is *not* invalid. While POSIX specified that it is based on C89, the original POSIX spec is silent on what the semantics is if a different C implementation is used - so at worst, the behavior of such applications is undefined. However, it is actually better than that: The C99 spec defines clearly what the semantics of all the functions are, and thus what the difference in behavior is. So for a specific POSIX application, the effect of compiling it with a C99 compiler is well-defined, and most applications should work just fine (if they compile at all). In a sense, Solaris is doing the same thing to its applications what you are criticizing here as Python doing to its extension modules: making claims of how compatible they are wrt. certain standards.] |
|||
msg82240 - (view) | Author: John Levon (movement) | Date: 2009-02-16 14:26 | |
Zooko, the patches are available here: http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/ > Then you were arguing that defining _XOPEN_SOURCE is wrong if the value Actually, I was saying that below that value simply won't work for many extensions. > What about setting the value to 600 or larger? Is that also wrong, > and if so, why? Practically: I don't know for sure about a larger value. As it is presumably very close to POSIX, it shouldn't significantly harm the ability to compile stuff. I can't say for sure. However, specifying _XOPEN_SOURCE is still unnecessary I believe. Theoretically: yes, it's wrong for Python to claim that all extensions are compliant to a particular XPG environment. If it helps, Martin, I can put you in touch with our standards expert. I'm far from such, so this might help assuage your doubts? |
|||
msg82274 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2009-02-16 21:43 | |
> Theoretically: yes, it's wrong for Python to claim that all extensions > are compliant to a particular XPG environment. It's somewhat unfortunate that Python makes such claims for extensions; primarily, these macros are defined for the compilation of Python itself. It would be possible to suppress them when compiling non-core extensions. However, experience tells that systems can break in surprising ways if the system headers are compiled with different defines. On some systems, the sizes and layouts of some types change under such macros, causing crashes when the extension sees a different size or layout than the Python core. > If it helps, Martin, I can put you in touch with our standards expert. > I'm far from such, so this might help assuage your doubts? I do feel this restrictiveness of the header files (wrt. C99) is arbitrary, and has no use. I think system vendors often interpret "feature selection macro" in a different way than it should be interpreted (i.e. instead of "make sure any program complying with the selected features compiles fine", the macros are interpreted as "reject any program that uses features beyond the selected ones"). So if there is a chance that Sun might reconsider this specific design choice, I'd love to talk with somebody - setting _XOPEN_SOURCE had worked fine since Solaris 7. Independent of such a discussion, I can grudgingly accept removal of _XOPEN_SOURCE on Solaris 5.{9,10,11} (?), to work-around this system limitation. |
|||
msg82338 - (view) | Author: John Levon (movement) | Date: 2009-02-17 14:30 | |
> However, experience tells that systems can break in surprising ways > if the system headers are compiled with different defines. This is indeed a reasonable concern (for which the best solution is dropping the defines in the Python compile). > I do feel this restrictiveness of the header files (wrt. C99) is > arbitrary, and has no use. Unfortunately, neither you, I, nor Sun can do anything at all about this fact. I've filed an RFE for you: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6806390 feature_tests.h could be laxer about C99 (It'll take a while to be visible externally.) Even if this happens, though, it's strictly a workaround: it is still a bug that Python sets these defines on Solaris: they do not mean what the Python build thinks they mean. It's effectively a Linux-ism. > setting _XOPEN_SOURCE had worked fine since Solaris 7. The particular change we're talking about (C99) went back in 2003: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=4760794 4760794 UNIX03: sys/feature_tests.h updates required to support SUSv3 and c99 so Python has been broken by this for all of the Solaris 10 lifecycle. |
|||
msg86934 - (view) | Author: James Andrewartha (trs80) | Date: 2009-05-02 08:38 | |
I'm jhbuilding GNOME on Solaris, and the attached patch fixes the problem for me, having compiled Python with it I can now compile dbus-python, pycairo and pyorbit against it. |
|||
msg93300 - (view) | Author: Tim Flechtner (tflecht) | Date: 2009-09-29 15:58 | |
Hi. I think I'm running into the problem discussed in this ticket. I'm trying to build pycairo-1.8.8 on Solaris 10 (x86) against python-2.6.2 using gcc-4.4.1. I'm afraid I'm out of my depth in understanding really what's going on, but the error I get looks like it fits the symptoms described here. Can anyone point me towards the patch to apply? I don't see it as available here (but I wouldn't be shocked if I'm just being blind). Thanks! -tim gmake[2]: Entering directory `/export/home/src/ThirdParty/gnu/pycairo-1.8.8/src' /bin/bash ../libtool --tag=CC --mode=compile /opt/app/g++lib6/gcc-4.4/bin/gcc -DHAVE_CONFIG_H -I. -I/opt/app/g++lib6/python-2.6/include/python2.6 -I/opt/app/g++lib6/python-2.6/include/python2.6 -I/opt/app/nonc++/release-1/include/cairo -DNDEBUG -g -O2 -std=c99 -Wall -fno-strict-aliasing -MT _cairo_la-cairomodule.lo -MD -MP -MF .deps/_cairo_la-cairomodule.Tpo -c -o _cairo_la-cairomodule.lo `test -f 'cairomodule.c' || echo './'`cairomodule.c libtool: compile: /opt/app/g++lib6/gcc-4.4/bin/gcc -DHAVE_CONFIG_H -I. -I/opt/app/g++lib6/python-2.6/include/python2.6 -I/opt/app/g++lib6/python-2.6/include/python2.6 -I/opt/app/nonc++/release-1/include/cairo -DNDEBUG -g -O2 -std=c99 -Wall -fno-strict-aliasing -MT _cairo_la-cairomodule.lo -MD -MP -MF .deps/_cairo_la-cairomodule.Tpo -c cairomodule.c -fPIC -DPIC -o .libs/_cairo_la-cairomodule.o In file included from /usr/include/limits.h:18, from /opt/app/g++lib6/gcc-4.4/lib/gcc/i386-pc-solaris2.10/4.4.1/include-fixed/limits.h:122, from /opt/app/g++lib6/gcc-4.4/lib/gcc/i386-pc-solaris2.10/4.4.1/include-fixed/syslimits.h:7, from /opt/app/g++lib6/gcc-4.4/lib/gcc/i386-pc-solaris2.10/4.4.1/include-fixed/limits.h:11, from /opt/app/g++lib6/python-2.6/include/python2.6/Python.h:19, from cairomodule.c:32: /opt/app/g++lib6/gcc-4.4/lib/gcc/i386-pc-solaris2.10/4.4.1/include-fixed/sys/feature_tests.h:341:2: error: #error "Compiler or options invalid for pre-UNIX 03 X/Open applications and pre-2001 POSIX applications" |
|||
msg93329 - (view) | Author: Zooko O'Whielacronx (zooko) | Date: 2009-09-29 20:43 | |
In msg82274 MvL wrote: "I can grudgingly accept removal of _XOPEN_SOURCE on Solaris 5.{9,10,11} (?), to work-around this system limitation." Great! That's what the attached patch does. So please apply that patch and then people can stop coming to this ticket saying that they couldn't compile their extension modules until they applied the patch. Also then the Solaris folks who build an installation of Python for Solaris can remove the perl script that searches for "#define _XOPEN_SOURCE" and replaces it with the empty string when building Python. |
|||
msg93400 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2009-10-01 11:06 | |
We're in the middle of a release so this will have to wait until we are done. Apart from that, is there a reason *not* to apply the patch? (it only seems to affect Solaris after all, and Solaris users here seem consensual that the patch should be applied) |
|||
msg94117 - (view) | Author: Zooko O'Whielacronx (zooko) | Date: 2009-10-16 00:39 | |
I just tried to port a library of mine -- zfec -- to Nexenta (the Ubuntu variant built on top of OpenSolaris). I hit this bug because the Nexenta folks don't use all the patches that are applied to Python by the Solaris folks. My port of my library is blocked until the Nexenta folks apply this patch to their package of Python: https://bugs.launchpad.net/python/+bug/452667 # can't compile Python modules that use C99 |
|||
msg97129 - (view) | Author: anders musikka (avl) | Date: 2010-01-02 14:28 | |
Just wanted to chip in my $.02: Defining _XOPEN_SOURCE in the python headers causes problems for Solaris. It also causes problems for Ubuntu Linux. Because _XOPEN_SOURCE is defined, Python.h must included first in any program under Ubuntu. Perhaps the right fix is to just not define _XOPEN_SOURCE at all? Which platforms disable important features when you don't define _XOPEN_SOURCE? Sometimes there are standards which nobody follows. In such cases it can be advantageous to "just do what everyone expects". |
|||
msg105223 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2010-05-07 19:21 | |
A question about the patch: in configure.in, after "if test $define_xopen_source = yes" (lines 395 and following), there's a bunch of Solaris-specific stuff. Shouldn't it be removed, given that _XOPEN_SOURCE isn't defined anymore under Solaris? |
|||
msg106675 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2010-05-28 15:46 | |
Thanks for the patch. Committed as r81582 and r81583. Antoine was right: subsequent references to Solaris needed to be removed also. |
|||
msg134790 - (view) | Author: David Kirkby (drkirkby) | Date: 2011-04-29 17:09 | |
Is there any progress on this? I see it is marked as Status: closed Resolution: accepted Stage: patch review That apparently means: ''There is a patch, but it needs reviewing or is in the process of being reviewed. This can be done by any triager as well as a core developer.'' So is there any movement on this, which will allow it to be fixed? I'm somewhat puzzled the Status is marked as "closed" when apparently the patch is still awaiting review. Dave |
|||
msg134791 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2011-04-29 17:14 | |
As noted by Martin above (he quoted the Subversion revision numbers), this was actually fixed. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:56:25 | admin | set | github: 45238 |
2011-04-29 17:14:32 | pitrou | set | resolution: accepted -> fixed messages: + msg134791 stage: patch review -> resolved |
2011-04-29 17:09:19 | drkirkby | set | nosy:
+ drkirkby messages: + msg134790 |
2010-11-25 15:36:58 | jcea | set | nosy:
+ jcea |
2010-05-28 15:46:51 | loewis | set | status: open -> closed resolution: accepted messages: + msg106675 |
2010-05-20 20:39:51 | skip.montanaro | set | nosy:
- skip.montanaro |
2010-05-07 19:21:47 | pitrou | set | stage: patch review messages: + msg105223 versions: + Python 3.1, Python 2.7, Python 3.2 |
2010-01-02 14:28:34 | avl | set | nosy:
+ avl messages: + msg97129 |
2009-10-16 00:39:04 | zooko | set | messages: + msg94117 |
2009-10-01 11:06:42 | pitrou | set | nosy:
+ pitrou messages: + msg93400 |
2009-09-29 20:43:00 | zooko | set | messages: + msg93329 |
2009-09-29 20:02:08 | skip.montanaro | set | nosy:
+ skip.montanaro |
2009-09-29 15:58:42 | tflecht | set | nosy:
+ tflecht messages: + msg93300 |
2009-05-02 08:38:44 | trs80 | set | nosy:
+ trs80 messages: + msg86934 |
2009-02-17 14:30:33 | movement | set | messages: + msg82338 |
2009-02-16 21:43:35 | loewis | set | messages: + msg82274 |
2009-02-16 14:26:59 | movement | set | messages: + msg82240 |
2009-02-16 08:18:04 | georg.brandl | set | nosy: - georg.brandl |
2009-02-16 08:17:03 | loewis | set | messages: + msg82223 |
2009-02-16 05:02:18 | zooko | set | messages: + msg82216 |
2009-02-16 02:59:26 | movement | set | messages: + msg82212 |
2009-02-16 02:06:39 | loewis | set | messages: + msg82203 |
2009-02-16 01:22:57 | movement | set | nosy:
+ movement messages: + msg82201 |
2009-02-02 16:35:02 | loewis | set | messages: + msg80949 |
2008-09-24 12:23:13 | zooko | set | messages: + msg73709 |
2007-09-02 19:14:52 | loewis | set | messages: + msg55588 |
2007-07-23 20:24:10 | zooko | create |