This looks like a recurring and somewhat sore topic.

For those of us that have been fighting the dreaded:

./Include/pyport.h:730:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."

when performing a 64 bit compile. I believe I have identified the problems. All of which are directly related to the Makefile(s) that are generated as part of the configure script. There does not seem to be anything wrong with the configure script or anything else once all of the Makefiles are corrected Python will build 64 bit

Although it is possible to pass the following environment variables to configure as is typical on most open source software:

  CC          C compiler command
  CFLAGS      C compiler flags
  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
              nonstandard directory <lib dir>
  CPPFLAGS    C/C++ preprocessor flags, e.g. -I<include dir> if you have
              headers in a nonstandard directory <include dir>
  CPP         C preprocessor

These flags are *not* being processed through to the generated Makefiles. This is where the problem is. configure is doing everything right and generating all of the necessary stuff for a 64 bit compile but when the compile is actually performed - the necessary CFLAGS are missing and a 32 bit compile is initiated.

Taking a close look at the first failure I found the following:

gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes  -I. -I./Include  -fPIC -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c

Where are my CFLAGS???

I ran the configure with:

CFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \
CXXFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \
LDFLAGS="-m64 -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \
./configure --prefix=/opt \
--enable-shared \
--libdir=/opt/lib/sparcv9

Checking the config.log and config.status it was clear that the flags were used properly as the configure script ran however, the failure is in the various Makefiles to actually reference the CFLAGS and LDFLAGS.

LDFLAGS is simply not included in any of the link stages in the Makefiles and CFLAGS is overidden by BASECFLAGS, OPT and EXTRA_CFLAGS!

Ah!

EXTRA_CFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \
make

Actually got the core parts to compile for the library and then failed to build the library because - LDFLAGS was missing from the Makefile for the library link stage - :-(

Close examination suggests that the OPT environment variable could be used to pass the necessary flags through from conifgure but this still did not help the link stage problems.

The fixes are pretty minimal to ensure that the configure variables are passed into the Makefile. My patch to the Makefile.pre.in is attached to this bug report.

Once these changes are made Python will build properly for both 32 and 64 bit platforms with the correct CFLAGS and LDFLAGS passed into the configure script.

BTW, while this bug is reported under a Solaris/gcc build the patches to Makefile.pre.in should fix similar build issues on all platforms.

Can you please report what the actual problem is that you got? I doubt it's the #error, as that error is generated by the preprocessor, yet your fix seems to deal with LDFLAGS only.

So please explain what command you invoked, what the actual output was, and what the expected output was.

OK, here is the synposis:

Run the configure with:

$ CFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \
CXXFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \
LDFLAGS="-m64 -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \
./configure --prefix=/opt \
--enable-shared \
--libdir=/opt/lib/sparcv9


$ make

gcc -pthread -c -fno-strict-aliasing -DNDEBUG    -I. -I./Include  -fPIC -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c
In file included from ./Include/Python.h:57,
                 from ./Modules/python.c:3:
./Include/pyport.h:730:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."

----

Cause:

Makefile.pre.in does not have substitutions that carry through the CFLAGS that were given to the configure script. In addition although LDFLAGS is carried in from the configure script it is not used in any of the link stages in the Makefile.

Minor issue. Makefile.pre.in also does not carry through the 'libdir' configure variable and should be referencing the other pre-defined configure variables.

Solution: See attached patches to Makefile.pre.in

File Added: Makefile.pre.in.patch

Hi,

I'd like to know the status of this issue. I'm having the same problems
here with PPC64, and the patch that Bob Atkins has sent works fine for
me too. Would you intend to apply this patch in upstream?

Thanks in advance.

> I'd like to know the status of this issue. I'm having the same problems
> here with PPC64, and the patch that Bob Atkins has sent works fine for
> me too. Would you intend to apply this patch in upstream?
> 
> Thanks in advance.

It's difficult to tell whether the patch can be applied until it is
reviewed. Until then, this issue will stay open.

Hi Martin,

Thanks for your quick answer. I'd like to know what can we do to push
this patch into upstream. Does the fact that the patch is posted in a
bug report (and not in a developer's mailing list) is slowing down the
reviewing process?

Regards.

> Thanks for your quick answer. I'd like to know what can we do to push
> this patch into upstream. Does the fact that the patch is posted in a
> bug report (and not in a developer's mailing list) is slowing down the
> reviewing process?

No, it's the lack of reviewers in the first place that is slowing all
patches, not just this one.

I'm skeptical about this patch. We already have BASECFLAGS, and blindly
copying  @CFLAGS@ into CFLAGS might break things if configure also
decides to add the same or conflicting flags.

In addition to setting BASECFLAGS, I think you can achieve the same
thing by setting CC to "gcc -O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9
-R/opt/lib/sparcv9" during configure.

Hi Martin,

Actually, I know that you can use CC to do it, but IMHO that's not the
correct approach. I understand too you concern about adding @CFLAGS@,
but I think the user should be able to define his/her own CFLAGS, and
this is not implemented yet. Do you agree with that?

Regards.

> Actually, I know that you can use CC to do it, but IMHO that's not the
> correct approach. I understand too you concern about adding @CFLAGS@,
> but I think the user should be able to define his/her own CFLAGS, and
> this is not implemented yet. Do you agree with that?

No, I don't agree this is not implemented. The user can set CC,
BASECFLAGS, and OPT, all of them contributing the CFLAGS.

Hi Martin,

This is what you get when you try to build a 64-bit Python on a biarch
machine (64-bit kernel, 32-bit userspace), using a gcc that generates
natively 32-bit objects (therefore, you *must* pass the '-m64' option
for the compiler):

#> ./configure --enable-shared --target=powerpc64-unknown-linux
BASECFLAGS='-m64'

<output generated by configure script>

#> make

gcc -pthread -c -m64 -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes  -I. -IInclude -I./Include  -fPIC
-DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c
In file included from Include/Python.h:57,
                 from ./Modules/python.c:3:
Include/pyport.h:761:2: error: #error "LONG_BIT definition appears wrong
for platform (bad gcc/glibc config?)."
make: *** [Modules/python.o] Error 1


As you can see, the compilation fails. Now, if I try this configure line:

#> ./configure --enable-shared --target=powerpc64-unknown-linux
BASECFLAGS='-m64' CFLAGS='-m64'

<output from configure>

#> make

Compilation goes well untill:

gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  
Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o
Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o
Parser/firstsets.o Parser/grammar.o Parser/pgen.o Objects/obmalloc.o
Python/mysnprintf.o Parser/tokenizer_pgen.o Parser/printgrammar.o
Parser/pgenmain.o -lpthread -ldl  -lutil -o Parser/pgen

As you can see, in this specific line we don't have the '-m64' flag,
what causes a bunch of errors (all of them due to the absence of '-m64'
flag). Ok, so I decided to try with LDFLAGS:

#> ./configure --enable-shared --target=powerpc64-unknown-linux
BASECFLAGS='-m64' CFLAGS='-m64' LDFLAGS='-m64'

<output from configure>

#> make

Now, the error happens when libpython.so is generated (and the reason is
the same: missing '-m64').

Well, now I have a few questions:

1) As you could see above, actually you need CFLAGS in order to compile
Python correctly. As far as I could investigate, the reason you need
this is because of the tests that are done by configure. Without the
CFLAGS, configure will think it's building a 32-bit Python, despite of
the '-m64' flag in BASECFLAGS. So, do we need to propagate CFLAGS
through Makefile or not? IMHO, we do.

2) Even with CFLAGS and BASECFLAGS set, the compilation fails. Using
LDFLAGS makes the compilation process continue a little more, but it
still doesn't solve the problem. AFAIK, the reason it doesn't solve the
problem is, again, because we are not propagating it through the
Makefile. Can you see any different reason? Also, should we propagate
LDFLAGS through Makefile? IMHO, we should.

Ohh, before I forget: compilation succeeds if we use only CC='gcc -m64'.
But again, I don't think this is a solution for this issue :-).

> This is what you get when you try to build a 64-bit Python on a biarch
> machine (64-bit kernel, 32-bit userspace), using a gcc that generates
> natively 32-bit objects (therefore, you *must* pass the '-m64' option
> for the compiler):

Or you install an additional, different, C compiler that defaults to
AMD64.

> 1) As you could see above, actually you need CFLAGS in order to compile
> Python correctly. As far as I could investigate, the reason you need
> this is because of the tests that are done by configure. Without the
> CFLAGS, configure will think it's building a 32-bit Python, despite of
> the '-m64' flag in BASECFLAGS. So, do we need to propagate CFLAGS
> through Makefile or not? IMHO, we do.

Not necessarily. I think you can achieve the same effect by specifying
CC="gcc -m64" to configure.

> 2) Even with CFLAGS and BASECFLAGS set, the compilation fails. Using
> LDFLAGS makes the compilation process continue a little more, but it
> still doesn't solve the problem. AFAIK, the reason it doesn't solve the
> problem is, again, because we are not propagating it through the
> Makefile. Can you see any different reason? Also, should we propagate
> LDFLAGS through Makefile? IMHO, we should.

Again, if you specify CC as above, you don't have to.

> Ohh, before I forget: compilation succeeds if we use only CC='gcc -m64'.
> But again, I don't think this is a solution for this issue :-).

Why not?

Regards,
Martin

Martin,

I've been quietly reading all of the back and forth regarding this problem.

Your suggestion for using the CC variable to fix the problem that I 
reported won't work - I already tried it and based on what others are 
reporting, you are beating a dead horse. Believe me I would rather not 
modify anyone's code in order to build something. The problem is that 
the CC variable doesn't fix the link stages and overall your autconf 
build is broken particularly as it relates to flowing variables down to 
sub builds.

I don't know why you are resisting this change. I took the time to 
report the bug, proposed a fix /_*and*_/ contributed the patch that 
would make the Python build process more standard relative to the vast 
majority of open source packages that use autoconf. I am glad to see 
that my patch appears to be generic enough that it works on other 
platforms as well. I didn't have to post a bug report let alone 
contribute a patch but, I believe strongly that is what open source is 
all about. As the maintainer you don't have to accept either the bug or 
the patch but, resisting without good cause will discourage further 
contributions - certainly from me because I won't waste my time 
submitting something when I know that a maintainer of a package is being 
closed minded. We do a lot of work with open source software here and it 
is our policy to contribute back to the community as much as possible. 
However, when we run into a brick wall we quickly give up because we 
can't justify the time and effort. As an example, we have completely 
suspended all contributions to the asterisk project. We operate a very 
large asterisk environment with a lot of fixes and improvements that I 
am sure lots of others would love to have but the maintainer's attitude 
was that a Sun Solaris platform was not important. What the maintainer 
doesn't know is that many of our fixes and changes affect /_*all*_/ 
platforms. So now they get nothing from us and the asterisk community as 
a whole is deprived of the benefits of our work. I also know that many 
others have also suspended contributions for the same reason and as a 
result the asterisk package suffers. The resistance on your part to 
recognizing this problem and a fix is unjustified.

Overall it improves the Python package if it can be built on more 
platforms in different ways - especially if it uses the standard 
autoconf approach that the majority of other open source packages use. 
Those of us that have to deal with building almost a hundred different 
packages for different platforms and architectures very much appreciate 
not having to deal with unnecessary exceptions or idiosyncrasies.

I don't care whether you incorporate the change or not - we certainly 
will continue to patch future versions of Python (we have a fully 
automated build process here) in order to produce a successful build. 
Clearly the genie is out of the bottle and others will also likely use 
the patch for their builds. Why not just make everyone happy and 
incorporate it or a reasonably similar approach that properly implements 
the flow of /_*all*_/ autoconf variables to the sub-builds of the Python 
package so that more people can take full advantage of Python on their 
64 bit platforms and also deal with whatever unique build requirements 
that they might have.

Sincerely,
Bob Atkins

Martin v. Löwis wrote:

>Martin v. Löwis <martin@v.loewis.de> added the comment:
>
>  
>
>>This is what you get when you try to build a 64-bit Python on a biarch
>>machine (64-bit kernel, 32-bit userspace), using a gcc that generates
>>natively 32-bit objects (therefore, you *must* pass the '-m64' option
>>for the compiler):
>>    
>>
>
>Or you install an additional, different, C compiler that defaults to
>AMD64.
>
>  
>
>>1) As you could see above, actually you need CFLAGS in order to compile
>>Python correctly. As far as I could investigate, the reason you need
>>this is because of the tests that are done by configure. Without the
>>CFLAGS, configure will think it's building a 32-bit Python, despite of
>>the '-m64' flag in BASECFLAGS. So, do we need to propagate CFLAGS
>>through Makefile or not? IMHO, we do.
>>    
>>
>
>Not necessarily. I think you can achieve the same effect by specifying
>CC="gcc -m64" to configure.
>
>  
>
>>2) Even with CFLAGS and BASECFLAGS set, the compilation fails. Using
>>LDFLAGS makes the compilation process continue a little more, but it
>>still doesn't solve the problem. AFAIK, the reason it doesn't solve the
>>problem is, again, because we are not propagating it through the
>>Makefile. Can you see any different reason? Also, should we propagate
>>LDFLAGS through Makefile? IMHO, we should.
>>    
>>
>
>Again, if you specify CC as above, you don't have to.
>
>  
>
>>Ohh, before I forget: compilation succeeds if we use only CC='gcc -m64'.
>>But again, I don't think this is a solution for this issue :-).
>>    
>>
>
>Why not?
>
>Regards,
>Martin
>
>_____________________________________
>Tracker <report@bugs.python.org>
><http://bugs.python.org/issue1628484>
>_____________________________________
>
>
>  
>

> Your suggestion for using the CC variable to fix the problem that I 
> reported won't work - I already tried it and based on what others are 
> reporting, you are beating a dead horse. Believe me I would rather not 
> modify anyone's code in order to build something. The problem is that 
> the CC variable doesn't fix the link stages

Why is that? It should work fine.

> and overall your autconf 
> build is broken particularly as it relates to flowing variables down to 
> sub builds.

This I don't understand. What is a sub build?

> I don't know why you are resisting this change. I took the time to 
> report the bug, proposed a fix /_*and*_/ contributed the patch that 
> would make the Python build process more standard relative to the vast 
> majority of open source packages that use autoconf.

I don't think that's the case. In autoconf, if you specify CFLAGS,
you expect that this overrides, not adds to, anything configure
computes on its own. This patch does not do that, i.e. it doesn't
allow you to override the settings that configure computes.

I'm concerned that the patch merely makes the entire setup more
complex, rather than solving an actual technical problem. That's
why I'm resisting to accept it.

Martin,

Martin v. Löwis wrote:

It does not when link stage specific options are required that are not 
valid for compilation stages.

My mistake. I was thinking of another package - you can scratch that 
comment.

There is no hard and fast rule that specifying the CFLAGS or other 
configure variables have to override /_*all*_/ of the compile flags that 
a developer chooses to use in their build process. Most open source 
packages use the configure CFLAGS, LDFLAGS, etc, variables to add to the 
compile and link flags and not completely override them. My patch 
follows the same philosophy that I have seen in the majority of other 
open source packages.

On the contrary - it does /_*not*_/ make it more complex and it 
/_*does*_/ provide a solution to a number of compile and link failures 
that have been reported over and over again. When I first posted this 
bug I read through the various forums and bug reports and I noticed 
quite a bit of complaining regarding this issue and I acknowledged that 
by noting in my report that what I was opening a bug report on what was 
clearly a sore subject within the Python community. To be specific the 
complaints were regarding the symptoms (unusual compile and link 
problems) related to this issue, not the specific cause of the problem 
which is why I filed the bug as I did.

The patch that I submitted will allow a build of Python to be performed 
the /_exact_/ same way it is today without any modifications to CFLAGS 
or LDFLAGS, etc. Those of us that /need/ to use these variables will be 
able to modify the build options to the configure script as necessary 
/_*without*_/ any complexity and without losing 'important' compile 
options that are currently generated by configure. All of your 
suggestions to use the CC variable in lieu of using the more appropriate 
CFLAGS, LDFLAGS, etc., introduce the very complexity and non-standard 
approach you are claiming you want top avoid.

Again, I don't see what your concerns are. My patch is a completely 
non-affecting change to the way Python is built today /_*but*_/ it does 
add the necessary fixes so that those that require specific compile and 
link options can specify them to the configure script.

In addition, while you are fixing the build/compile options, I noticed 
that you are missing the -fPIC compile option. Without it you can't link 
shared libraries and the build fails. You should probably be including 
-fPIC as a standard compile option so it is not required to be passed 
into the build via the configure script.

---
Bob

Hi Martin,

On Mon, 2008-04-14 at 20:04 +0000, Martin v. Löwis wrote:
> Martin v. Löwis <martin@v.loewis.de> added the comment:
> 
> > This is what you get when you try to build a 64-bit Python on a biarch
> > machine (64-bit kernel, 32-bit userspace), using a gcc that generates
> > natively 32-bit objects (therefore, you *must* pass the '-m64' option
> > for the compiler):
> 
> Or you install an additional, different, C compiler that defaults to
> AMD64.

I cannot do that. Actually, even if I could, I don't think this is the
best way to handle this *Python*'s problem.

> > 1) As you could see above, actually you need CFLAGS in order to compile
> > Python correctly. As far as I could investigate, the reason you need
> > this is because of the tests that are done by configure. Without the
> > CFLAGS, configure will think it's building a 32-bit Python, despite of
> > the '-m64' flag in BASECFLAGS. So, do we need to propagate CFLAGS
> > through Makefile or not? IMHO, we do.
> 
> Not necessarily. I think you can achieve the same effect by specifying
> CC="gcc -m64" to configure.

I know that. But the purpose of CC flag is to define a compiler to be
used in the compilation, and not to specify compiler flags (for that, we
have CFLAGS).

> > Ohh, before I forget: compilation succeeds if we use only CC='gcc -m64'.
> > But again, I don't think this is a solution for this issue :-).
> 
> Why not?

See above.

Regards,

On Tue, 2008-04-15 at 02:01 +0000, Bob Atkins wrote:

> I don't know why you are resisting this change. I took the time to 
> report the bug, proposed a fix /_*and*_/ contributed the patch that 
> would make the Python build process more standard relative to the vast 
> majority of open source packages that use autoconf. I am glad to see 
> that my patch appears to be generic enough that it works on other 
> platforms as well. I didn't have to post a bug report let alone 
> contribute a patch but, I believe strongly that is what open source is 
> all about. As the maintainer you don't have to accept either the bug or 
> the patch but, resisting without good cause will discourage further 
> contributions - certainly from me because I won't waste my time 
> submitting something when I know that a maintainer of a package is being 
> closed minded. We do a lot of work with open source software here and it 
> is our policy to contribute back to the community as much as possible. 
> However, when we run into a brick wall we quickly give up because we 
> can't justify the time and effort. As an example, we have completely 
> suspended all contributions to the asterisk project. We operate a very 
> large asterisk environment with a lot of fixes and improvements that I 
> am sure lots of others would love to have but the maintainer's attitude 
> was that a Sun Solaris platform was not important. What the maintainer 
> doesn't know is that many of our fixes and changes affect /_*all*_/ 
> platforms. So now they get nothing from us and the asterisk community as 
> a whole is deprived of the benefits of our work. I also know that many 
> others have also suspended contributions for the same reason and as a 
> result the asterisk package suffers. The resistance on your part to 
> recognizing this problem and a fix is unjustified.

Bob just took the words from my mouth. Martin, with all respect, your
resistance in accepting this patch is making things much harder that
they really are. The main point here is that this pacth actually
*doesn't* break anything in Python!

Please, take a time to consider our proposal.

Thanks,

Just to repeat my concern about this patch: It gives the impression of
supporting CFLAGS, but doesn't really. There *is* a hard rule that
CFLAGS given to configure should override any options passed to
configure, and there is a reason for that: configure may guess
incorrectly, and may add options that actually conflict with the ones
that you are trying to pass in. For example, for Darwin, we add
-arch ppc -arch i386, so there is no way to not build a universal
binary anymore. Likewise, on SCO_SV, configure adds -Ki486 -DSCO5,
which may well be incorrect, and there would be no way to correct that.

Can you please agree that above description is factual wrt. the patch
you propose?

Therefore, I claim that this makes things more complex, and doesn't
solve an actual problem.

Martin,

On solaris 10 x86, this patch makes it possible to build python 2.5.x.
Without it, there is no way for the automated build to work. I believe
that your characterization of it as "Therefore, I claim that this makes
things more complex, and doesn't solve an actual problem." is strangely
disconnected from the facts that have been presented to you.

So, since this patch allows python to be built 64-bit on a biarch
system, and without it, the build doesn't work, what would need to
change so that you/the python maintainers would accept a fix? Assuming
this patch isn't it, what needs to change? I think that in this entire
conversation a set of viable parameters haven't been presented.

As it is, python is ridiculously difficult to build for my company's
environment which has separate packages in separate directories (eg
/usr/local/amd64/expat/ for expat, /usr/local/amd64/gnu/lib for readline
and ncurses, etc.)

Thanks,

-Peter

> So, since this patch allows python to be built 64-bit on a biarch
> system, and without it, the build doesn't work

This is simply not true. I can build Python 2.5 just fine for 64-bit
SPARC, using gcc, with

CC="gcc -m64" ./configure
make

Or, using SunPRO, with

CC="cc -m64" ./configure
make

I tested it myself, and it successfully builds a Python executable
(For gcc, the extension modules fail to load because it picks up
the wrong libgcc_s, which I believe is a gcc installation bug. For
SunPRO, the extension modules also build fine).

So before anything is fixed, I'd like to see an actual, reliable,
reproducable error report. The subject of this report (Python 2.5
64 bit compile fails on Solaris 10 with gcc) is not reproducable,
so I'm tempted to close this report as "works for me".

Martin,

Your method is just flat wrong - equivalent to using a sledgehammer. The 
libraries fail to link not because gcc install is wrong but because the 
-m64 flag needs to be passed to the linker. Your method just fixes the 
compilation stage. You are blaming the gcc installation for your problem 
when in fact the /_*rest*_/ of the open source world builds just fine 
using gcc when the proper flags are passed at the proper stages of the 
build.

This situation is so sad to see. This isn't the way open source 
development is supposed to be. Basically _/*you*/_ alone are the 
gatekeeper and you are refusing to deal with what is a very simple problem.

To be sure I really don't care anymore - we will continue to apply the 
patches that we have to fix the build problem however, we will also 
actively discourage the use of Python for our customers and all future 
development/deployment since it is obvious that as the maintainer you 
are completely closed minded and uncooperative when it comes to fixing 
things.

My suggestion for the rest of the Python community is to branch and 
re-host the Python project elsewhere if anyone wants to see it move forward.

> Your method is just flat wrong - equivalent to using a sledgehammer. The 
> libraries fail to link not because gcc install is wrong but because the 
> -m64 flag needs to be passed to the linker.

And indeed, the flag *is* passed to the linker. Python links with CC
unless specified otherwise, which will be "gcc -m64". That ought to
work, and it does work when linking Python itself.

> Your method just fixes the compilation stage.

Not true.

> To be sure I really don't care anymore - we will continue to apply the 
> patches that we have to fix the build problem however, we will also 
> actively discourage the use of Python for our customers and all future 
> development/deployment since it is obvious that as the maintainer you 
> are completely closed minded and uncooperative when it comes to fixing 
> things.

I'm sorry you feel that way. Feel free to bring up the issue on
python-dev if you think you are being ignored. I assure you this is
not the case - but I want to see the problem first before accepting
fixes.

Just to demonstrate there is really a problem with the gcc installation
(gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath)), here is the
linker line:

gcc -m64 -shared
build/temp.solaris-2.10-sun4u-2.5/net/tb0/home/loewis/25/Modules/_struct.o
-L/usr/local/lib -o build/lib.solaris-2.10-sun4u-2.5/_struct.so

and here is what gcc actually invokes (printed with -v):

/usr/sfw/libexec/gcc/sparc-sun-solaris2.10/3.4.3/collect2 -V -G -dy -z
text -Y P,/usr/lib/sparcv9 -Qy -o
build/lib.solaris-2.10-sun4u-2.5/_struct.so
/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crti.o
/usr/ccs/lib/sparcv9/values-Xa.o
/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crtbegin.o
-L/usr/local/lib -L/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9
-L/usr/ccs/lib/sparcv9
-L/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/../../../sparcv9
-L/lib/sparcv9 -L/usr/lib/sparcv9
build/temp.solaris-2.10-sun4u-2.5/net/tb0/home/loewis/25/Modules/_struct.o
-R/usr/sfw/lib -lgcc_s_sparcv9 -R/usr/sfw/lib -lgcc_s_sparcv9
/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crtend.o
/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crtn.o

As you can see, it's picking up -lgcc_s_sparcv9 (from
/usr/sfw/lib/sparcv9/libgcc_s_sparcv9.so), so it does link with the
64-bit libgcc_s.so.1. However, when then trying to import the module, it
complains 

ImportError: ld.so.1: python: fatal: /usr/sfw/lib/libgcc_s.so.1: wrong
ELF class: ELFCLASS32

This is due to the option -R/usr/sfw/lib, which should have been
-R/usr/sfw/lib/sparcv9. Fixing the specs file to change the -R option
actually fixes that problem.

So please don't claim that I did something wrong when there is really a
bug in sfw version of gcc.

I rest my case - you found /_*one*_/ of the problems which you are 
blaming on gcc but in fact is not gcc's fault. You /_*must*_/ specify 
the correct -L and -R paths to the various alternate 64 bit libs. Don't 
expect the compiler to figure it out. Of course you can also fix the 
problem by changing the defaults to the system link/loader and I'm sure 
other non-standard methods.

The bottom line is that I just don't know how to describe daylight to a 
blind man....

FYI, we are using gcc 4.1.1 - same problem and we aren't using or 
relying on just the /usr/sfw tree sine much of what comes with Solaris 
isn't 64 bit we have had to build our own libs which are kept in /opt/lib

---
Bob

Martin v. Löwis wrote:
> Martin v. Löwis <martin@v.loewis.de> added the comment:
>
> Just to demonstrate there is really a problem with the gcc installation
> (gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath)), here is the
> linker line:
>
> gcc -m64 -shared
> build/temp.solaris-2.10-sun4u-2.5/net/tb0/home/loewis/25/Modules/_struct.o
> -L/usr/local/lib -o build/lib.solaris-2.10-sun4u-2.5/_struct.so
>
> and here is what gcc actually invokes (printed with -v):
>
> /usr/sfw/libexec/gcc/sparc-sun-solaris2.10/3.4.3/collect2 -V -G -dy -z
> text -Y P,/usr/lib/sparcv9 -Qy -o
> build/lib.solaris-2.10-sun4u-2.5/_struct.so
> /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crti.o
> /usr/ccs/lib/sparcv9/values-Xa.o
> /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crtbegin.o
> -L/usr/local/lib -L/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9
> -L/usr/ccs/lib/sparcv9
> -L/usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/../../../sparcv9
> -L/lib/sparcv9 -L/usr/lib/sparcv9
> build/temp.solaris-2.10-sun4u-2.5/net/tb0/home/loewis/25/Modules/_struct.o
> -R/usr/sfw/lib -lgcc_s_sparcv9 -R/usr/sfw/lib -lgcc_s_sparcv9
> /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crtend.o
> /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/sparcv9/crtn.o
>
> As you can see, it's picking up -lgcc_s_sparcv9 (from
> /usr/sfw/lib/sparcv9/libgcc_s_sparcv9.so), so it does link with the
> 64-bit libgcc_s.so.1. However, when then trying to import the module, it
> complains 
>
> ImportError: ld.so.1: python: fatal: /usr/sfw/lib/libgcc_s.so.1: wrong
> ELF class: ELFCLASS32
>
> This is due to the option -R/usr/sfw/lib, which should have been
> -R/usr/sfw/lib/sparcv9. Fixing the specs file to change the -R option
> actually fixes that problem.
>
> So please don't claim that I did something wrong when there is really a
> bug in sfw version of gcc.
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue1628484>
> _______________________________________
>
>
>

Hi Bob, thank you for your patch. I spent hours on Solaris 10 SPARC to
get almost the same analysis. Just a detail, I ended up patching
$LDFLAGS in the SunOS 5 part in the configure.in file (like other
architectures like Darwin have set their LDFLAGS there, too).

Now I'm hoping that the Python team will accept this patch. Otherwise it
will be very hard and complicated for everybody else to set up a build
process for a 64-bit Python together with 32-bit Python on Solaris.

Perhaps we can get some movement regarding this problem again, as it
also applies to other platforms that require special gcc options for the
compiler and linker.

A common case where such settings were needed is Mac OS X - in the case
of building universal binaries. Since this was too tedious to get right,
Python 2.5 introduced new configure options to simplify this. In Python
2.4, you had to configure Python using these  configure options to get a
universal build:

BASECFLAGS="-arch ppc -arch i386 -isysroot
/Developer/SDKs/MacOSX10.4u.sdk" LDFLAGS="-arch i386 -arch ppc -isysroot
/Developer/SDKs/MacOSX10.4u.sdk"

... and that shows part of the problem with the Python configure/make
system: it simply doesn't follow the standards used in other OSS
software of passing through CFLAGS et al. to all subsequent compiler and
linker calls.

In order to get compiler options passed through, you have to set BASECFLAGS.

For linker options, you may have some luck with using LDFLAGS, but not
always, since e.g. the configure script may sometimes add LDFLAGS to
LDSHARED (e.g. on Mac OS X using the universal binary options), which
then results in the options to show up twice on the linker line. Using
LDSHARED directly instead then helps.

As a result, simply adding CFLAGS and LDFLAGS to a few more targets
definitions in the Makefile will likely cause more trouble on other
platforms and in other situations.

Overall, the whole configure/Makefile system for defining compiler and
linker options has gotten somewhat messy over the years and much of it
is not documented (at least I'm not aware of any such documentation
apart from the ticket and checkin messages related to the various changes).

I think a first step in the right direction would be to make sure that
LDSHARED never automagically gets additional values from LDFLAGS. Then
we could at least add LDFLAGS to all targets that use LDSHARED as linker
command for shared libs.

As second step, I think that the CFLAGS environment variable passed to
configure should be made to init the BASECFLAGS Makefile variable, since
that's what the user would expect (if he knew how the system works).

Only about LDFLAGS.
The python build system evolve and executable and libraries are build
with LDFLAGS as is. So except passing LDFLAGS to setup.py rest of Bob
Atkins patch is in the makefile.

As part of issue 4010 I post a patch "py-issue-4010.patch" (thanks to
John P. Speno that point for quote of LDFLAGS), i.e. same as Bob patch.
The rest of "py-issue-4010.patch" is clean up of configure.in (avoid 
options doubling on BSD based plaforms) and setup.py scripts.

[...]
> As second step, I think that the CFLAGS environment variable passed to
> configure should be made to init the BASECFLAGS Makefile variable, since
> that's what the user would expect (if he knew how the system works).

I still think that such a patch would be flawed, because it *still*
wouldn't follow the standards used in other OSS software, where setting
CFLAGS overrides *ALL* settings that configure may have come up with.

So if a CFLAGS environment variables is to be considered, it needs to
be considered correctly.

Martin v. Löwis wrote:
> 
> Martin v. Löwis <martin@v.loewis.de> added the comment:
> 
> [...]
>> As second step, I think that the CFLAGS environment variable passed to
>> configure should be made to init the BASECFLAGS Makefile variable, since
>> that's what the user would expect (if he knew how the system works).
> 
> I still think that such a patch would be flawed, because it *still*
> wouldn't follow the standards used in other OSS software, where setting
> CFLAGS overrides *ALL* settings that configure may have come up with.
> 
> So if a CFLAGS environment variables is to be considered, it needs to
> be considered correctly.

Fair enough, then let's do that.

If we go down that road, we'd have to remove BASECFLAGS, OPT and
EXTRA_CFLAGS and replace it with the single standard CFLAGS
variable, or use an approach similar to the one taken for CPPFLAGS
(ie. we allow these extra variables to further customize a CFLAGS
setting):

CPPFLAGS= @BASECFLAGS@ @OPT@ @EXTRA_CFLAGS@ @CFLAGS@

FWIW, the reason behind the extra variables is not really clear. They only
appear to factor out different aspects of the same thing:

r38847 | bcannon | 2005-04-25 00:26:38 +0200 (Mon, 25 Apr 2005) | 6 lines

Introduced EXTRA_CFLAGS as an environment variable used by the Makefile.  Meant
to be used for flags that change binary compatibility.

r30490 | montanaro | 2003-01-01 21:07:49 +0100 (Wed, 01 Jan 2003) | 10 lines

Split OPT make variable into OPT and BASECFLAGS.  The latter contains those
compiler flags which are necessary to get a clean compile.  The former is
for user-specified optimizer, debug, trace fiddling.  See patch 640843.

BTW: I've checked the use of LDFLAGS - this is added to LDSHARED
on Mac OS X, FreeBSD, OpenBSD and NetBSD. Perhaps this is something
special needed on BSDish system ?!

[Adding Jack Jansen to the nosy list, since he added the LDFLAGS parts
for Mac OS X]

Jack, could you please comment on why the LDFLAGS are added to LDSHARED
by configure, rather than using LDFLAGS as extra argument to LDSHARED ?

Thanks.

I see that Martin's broken record still hasn't changed. I had warm,
nostalgic feelings as I re-read this thread. It is so sad to see that
this matter remains unresolved almost 3 years after I filed this bug.

As usual Martin is just flat wrong in his insistence that a defined
CFLAGS must overide any generated CFLAGS by configure to be consistent
with other OSS. But of course that is just his excuse for not accepting
this bug and fix. If it wasn't this assertion then he would find
something else equally absurd.

Does anyone know the meaning of NIH?

I'll bet that this matter will never be resolved until the Python
community simply takes Python and re-hosts it somewhere else as
OpenPython. As long as Martin is the gatekeeper you can be assured that
this bug (or any bug) that he doesn't agree with will never be accepted
or fixed and if he does accept this bug - he will insist on doing it the
wrong way (by overriding CFLAGS) to prove he was 'right' all along -
that this bug fix will break more than it fixes.

Keep up the good work Martin. I am expecting that this bug will continue
to provide me with nostalgic memories well into my retirement...  ;-)

Martin isn't the gatekeeper, it's just that few people are really
motivated in solving tedious configuration-related problems, especially
when there are diverging concerns (legacy, habits, compatibility, etc.)
to take into account.

That said, I think the current CFLAGS situation is counter-intuitive and
would deserve being fixed.

> [...] because it *still*
> wouldn't follow the standards used in other OSS software, where setting
> CFLAGS overrides *ALL* settings that configure may have come up with.

Martin, can you please elaborate on this? I never heard of such
"standards" in OSS.

Bob Atkins wrote:
> 
> As usual Martin is just flat wrong in his insistence that a defined
> CFLAGS must overide any generated CFLAGS by configure to be consistent
> with other OSS. But of course that is just his excuse for not accepting
> this bug and fix. If it wasn't this assertion then he would find
> something else equally absurd.

I don't think so... we have to support multiple platforms
and doing so is rather difficult in the light of different
requirements and often missing ability to actually test on the
various platforms.

The configure/make system we have in Python has grown a lot over
the years and many people have contributed to it. As a result, it's
not as clean as it could be and there are many aspects that require
inside knowledge about the platforms.

Martin is right in saying that a CFLAGS environment variable has
to override the value determined by configure... see e.g.

http://www.gnu.org/software/hello/manual/autoconf/Preset-Output-Variables.html

However, the situation is a little more complex: CFLAGS should
normally *not* be used by the Makefile for things that configure
finds or regards as not user customizable.

Unfortunately, Python's Makefile does not work that way. It
builds its own CFLAGS value out of a combination of other
variables.

It should really be building a new variable based on CFLAGS and
the other variables and then use that in the build process.

The PY_CFLAGS variable appears to be a start in that direction,
though it's not being followed through.

What makes the situation even more complex is that C extensions
built with distutils will also use these variables for compilation,
so any changes to the way the variables work will have direct effect
on whether or not extensions build out of the box.

3 years and counting while everyone rings their hands and debates this
trivial issue.

3 years and counting while hundreds of builders encounter this problem
wasting countless of hours troubleshooting, possibly re-reporting the
problem.

Software is not a religion - but many software developers believe it is.

This issue could have been solved 3 years ago and more time spent on
other issues that really matter. Or you can spend the next 3 years
debating the fanatically correct way to solve this problem.

My money is that the fanatically 'correct' method will be implemented
that will require hundreds of lines of code, possibly re-engineering the
entire build process, introducing more problems and take a few more
years to implement and release.

Bob Atkins wrote:
> My money is that the fanatically 'correct' method will be implemented
> that will require hundreds of lines of code, possibly re-engineering the
> entire build process, introducing more problems and take a few more
> years to implement and release.

Well, I guess that's not your problem anymore... you have your patch
and it works for you and perhaps a few others as well. That's fine
for the time being.

Without knowing the impact of the generic approach you've taken
in your patch we simply cannot just apply it. If you can prove that
the patch doesn't break other platforms or configuration setups,
that would help a lot.

> Jack, could you please comment on why the LDFLAGS are added to 
LDSHARED
> by configure, rather than using LDFLAGS as extra argument to LDSHARED 
?

Because this worked, no deep reason. The initial framework builds were a 
big hack, because they were neither static nor shared builds (because 
the extensions were linked against the framework), so I had to find 
something that worked while hoping I wouldn't break too much on other 
platforms.

In case anyone is interested in my opinion: I would scratch the whole 
configure/make suite and rebuild it from scratch. As others here have 
noticed, the OPT/EXTRA_CFLAGS pattern that Python adhered to has lost 
out the the CFLAGS/LDFLAGS pattern, and more such things. And this is 
important if people want to do recursive builds.

But: it's a major undertaking to get this working, especially if you 
don't want to pull in libtool:-(

> Without knowing the impact of the generic approach you've taken
> in your patch we simply cannot just apply it. If you can prove that
> the patch doesn't break other platforms or configuration setups,
> that would help a lot.

I was able to build Python 2.5 on Solaris 10 Sparc, Mac OS X PPC, Linux
PPC/Intel, all 32bit and 64bit, shared and static, only with Bob's help.

It's not a proof. It's not mathematical correct. But it works. Grab all
your avalaible test platforms and try for yourself what Bob's patch will
'break', and report it. 

Sorry, but that meta discussions about correct builds are not what a bug
report should be used for. Such improvements are up to developer forums
where you can design "correct" Python build scripts and discuss them
over and over again.

On Fri, Oct 30, 2009 at 09:31:38PM +0000, J??rg Prante wrote:
> 
> J??rg Prante <joergprante@gmx.de> added the comment:
> 
> > Without knowing the impact of the generic approach you've taken
> > in your patch we simply cannot just apply it. If you can prove that
> > the patch doesn't break other platforms or configuration setups,
> > that would help a lot.
> 
> I was able to build Python 2.5 on Solaris 10 Sparc, Mac OS X PPC, Linux
> PPC/Intel, all 32bit and 64bit, shared and static, only with Bob's help.

Ditto for python 2.5 on Solaris 10 x86 64-bit.  It was simply
impossible without these patches. 

> It's not a proof. It's not mathematical correct. But it works. Grab all
> your avalaible test platforms and try for yourself what Bob's patch will
> 'break', and report it. 
>
> Sorry, but that meta discussions about correct builds are not what a bug
> report should be used for. Such improvements are up to developer forums
> where you can design "correct" Python build scripts and discuss them
> over and over again.

Agreed. +1 from me if it counts for anything (which it probably doesn't).

-Peter

First, the current patch doesn't apply cleanly to trunk. The following
patch should be ok (some of the changes of the original patch apparently
have been committed separately in the meantime).

Second, the patch allows me to do a 32-bit build (under 64-bit Linux) by
doing:
  CFLAGS=-m32 LDFLAGS=-m32 ./configure
rather than:
  CC="gcc -m32" ./configure
However, if I omit LDFLAGS it doesn't work, I don't know if it's intended.

Third, while the 32-bit build does work, the shared objects are still
placed in a directory called "lib.linux-x86_64-2.7", which I suppose is
wrong:

$ ./python
Python 2.7a0 (trunk:75966:75967M, Oct 30 2009, 22:55:18) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import _socket
>>> _socket.__file__
'/home/antoine/cpython/__svn__/build/lib.linux-x86_64-2.7/_socket.so'

$ file /home/antoine/cpython/__svn__/build/lib.linux-x86_64-2.7/_socket.so
/home/antoine/cpython/__svn__/build/lib.linux-x86_64-2.7/_socket.so: ELF
32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically
linked, not stripped

Marc-Andre,
Thanks for the reference but what about to open manual for AC_PROG_CC ?



Antoine,
please don't mess kind of cross compilation into this thread.


About patches:
Change of libdir are subject to other requests - require changes in
distutils - out of scope.

About CFLAGS : 
To ignore options like -g -O2 set by AC_PROG_CC just enclose macro in
py_save_CFLAGS ... CFLAGS=$py_save_CFLAGS. Тhe python build system use
own options OPT to set -g -O3 and etc. Please see comments in configure
for OPT.

About LDFLAGS with passing to setup.py (last place without it) is good
to remove all other references as I do in other issue . I won't update
my patch to apply cleanly to trunk if there is no interest.

Antoine Pitrou wrote:
> Second, the patch allows me to do a 32-bit build (under 64-bit Linux) by
> doing:
>   CFLAGS=-m32 LDFLAGS=-m32 ./configure
> rather than:
>   CC="gcc -m32" ./configure
> However, if I omit LDFLAGS it doesn't work, I don't know if it's intended.

Without the patch,

 BASECFLAGS=-m32 LDFLAGS=-m32 ./configure

should work the same way.

LDFLAGS defines the linker options, CFLAGS the compiler options,
and since both tools have to know that you're generating 32-bit code,
you have to pass the option to both env vars.

> Third, while the 32-bit build does work, the shared objects are still
> placed in a directory called "lib.linux-x86_64-2.7", which I suppose is
> wrong:

That's a side-effect of distutils using os.uname() for determining
the platform. It doesn't know that you're actually telling the
compiler to build a 32-bit binary.

On some platforms you can use these commands to emulate 32- or
64-bit environments:

$ linux32 python -c 'import os; print os.uname()'
('Linux', 'newton', '2.6.22.19-0.4-default', '#1 SMP 2009-08-14 02:09:16 +0200', 'i686')
$ linux64 python -c 'import os; print os.uname()'
('Linux', 'newton', '2.6.22.19-0.4-default', '#1 SMP 2009-08-14 02:09:16 +0200', 'x86_64')

> Antoine,
> please don't mess kind of cross compilation into this thread.

This is not cross-compilation, a 32-bit binary will run fine on a x86-64
system.

Roumen Petrov wrote:
> 
> Roumen Petrov <bugtrack@roumenpetrov.info> added the comment:
> 
> Marc-Andre,
> Thanks for the reference but what about to open manual for AC_PROG_CC ?

Could you please elaborate a bit ?

> Antoine,
> please don't mess kind of cross compilation into this thread.

That was just an example of how CFLAGS can be used.

However, with the patch you still don't get complete control
of the C compiler flags - which is what the env var should
be all about.

The reason is that the actually used CFLAGS argument then
becomes a combination of these env vars/settings:

CFLAGS=		$(BASECFLAGS) @CFLAGS@ $(OPT) $(EXTRA_CFLAGS)

To regain control over CFLAGS, you'd have to define these
env vars:

CFLAGS, BASECFLAGS, OPT, EXTRA_CFLAGS

That's not really how it should be... (see the autoconf
reference URL I posted).

> About patches:
> Change of libdir are subject to other requests - require changes in
> distutils - out of scope.

True.

> About CFLAGS : 
> To ignore options like -g -O2 set by AC_PROG_CC just enclose macro in
> py_save_CFLAGS ... CFLAGS=$py_save_CFLAGS. Тhe python build system use
> own options OPT to set -g -O3 and etc. Please see comments in configure
> for OPT.

In the early days we only allowed customization of the optimization
settings when compiling Python, nothing more. That's where OPT
originated.

Things started to get complicated when the recursive make process
was flattened starting in Python 2.1.

> About LDFLAGS with passing to setup.py (last place without it) is good
> to remove all other references as I do in other issue . I won't update
> my patch to apply cleanly to trunk if there is no interest.

Could you provide an issue number ?

Jack Jansen wrote:
> 
> Jack Jansen <jackjansen@users.sourceforge.net> added the comment:
> 
>> Jack, could you please comment on why the LDFLAGS are added to 
> LDSHARED
>> by configure, rather than using LDFLAGS as extra argument to LDSHARED 
> ?
> 
> Because this worked, no deep reason. The initial framework builds were a 
> big hack, because they were neither static nor shared builds (because 
> the extensions were linked against the framework), so I had to find 
> something that worked while hoping I wouldn't break too much on other 
> platforms.

Well, it does break on Mac OS X (and only there) now, since for
universal builds, the Mac gcc complains if you pass in the
-sysroot option more than once :-)

I was under the impression that the "-bundle" option was
needed as last linker option and that this was the reason
for adding LDFLAGS directly into LDSHARED.

But if there are no deep reasons, then I'd suggest to not add
LDFLAGS to LDSHARED anymore and instead just use it normally.

> In case anyone is interested in my opinion: I would scratch the whole 
> configure/make suite and rebuild it from scratch. As others here have 
> noticed, the OPT/EXTRA_CFLAGS pattern that Python adhered to has lost 
> out the the CFLAGS/LDFLAGS pattern, and more such things. And this is 
> important if people want to do recursive builds.

Even though we do have a flat Makefile now, there still is
some recursion left: Python uses distutils to build most of the
included C extension modules.

Part of the patch targets exactly this recursion: LDFLAGS
is currently not being passed on to the shared mod build
sub-process.

> But: it's a major undertaking to get this working, especially if you 
> don't want to pull in libtool:-(

What if we just remove all the extra cruft and just use
CFLAGS ?

LDFLAGS is not such a mess. It just needs to be propagated
to all parts of the process.

Mark issue is 4010 (see message #msg94686 above) .

About the control of the flags :) ... the Bob's post "... method will be
implemented that will require hundreds of lines of code ..." is true.

Order $(BASECFLAGS) @CFLAGS@ $(OPT) $(EXTRA_CFLAGS) look good as first
start with project CFLAGS followed by env. CFLAGS and why will read
README file for OPT and EXTRA_CFLAGS ?

> Martin, can you please elaborate on this? I never heard of such
> "standards" in OSS.

MAL already gave the link. From the link:

Sometimes package developers are tempted to set user variables such as
CFLAGS because it appears to make their job easier. However, the package
itself should never set a user variable, particularly not to include
switches that are required for proper compilation of the package. Since
these variables are documented as being for the package builder, that
person rightfully expects to be able to override any of these variables
at build time.

> > Martin, can you please elaborate on this? I never heard of such
> > "standards" in OSS.
> 
> MAL already gave the link. From the link:
> 
> Sometimes package developers are tempted to set user variables such as
> CFLAGS because it appears to make their job easier. However, the package
> itself should never set a user variable, particularly not to include
> switches that are required for proper compilation of the package. Since
> these variables are documented as being for the package builder, that
> person rightfully expects to be able to override any of these variables
> at build time.

So one day package builder by right will set CFLAGS to make their job
easier and python build system will not ignore it.

Marc-Andre,

on 64-bit Ubuntu, this method ...

BASECFLAGS=-m32 LDFLAGS=-m32 ./configure

... results in a gcc segfault:


gcc -pthread -m32 -Xlinker -export-dynamic -o python \
                        Modules/python.o \
                        libpython3.2.a -lpthread -ldl  -lutil   -lm  
Segmentation fault
make: *** [sharedmods] Error 139



Antoine's patch and command line work.

Stefan Krah wrote:
> 
> Stefan Krah <stefan-usenet@bytereef.org> added the comment:
> 
> Marc-Andre,
> 
> on 64-bit Ubuntu, this method ...
> 
> BASECFLAGS=-m32 LDFLAGS=-m32 ./configure
> 
> ... results in a gcc segfault:
> 
> 
> gcc -pthread -m32 -Xlinker -export-dynamic -o python \
>                         Modules/python.o \
>                         libpython3.2.a -lpthread -ldl  -lutil   -lm  
> Segmentation fault
> make: *** [sharedmods] Error 139
> 
> 
> 
> Antoine's patch and command line work.

Just to get an idea of why the above fails, could you provide the
line from the build with Antoine's patch applied ?!

I starting to think that we should apply Antoine's patch for 2.7/3.2
and leave the CFLAGS cleanup to some other release.

The builds are almost identical, so I attach a diff of the build output.
For both builds, I used a fresh Python-3.1.1 directory. This looks
suspicious:

-checking whether va_list is an array... yes
+checking whether va_list is an array... no


For completeness' sake: "-fno-strict-aliasing -m32" were reversed in the
BASECFLAGS case, so I edited the BASECFLAGS output for the purposes of
creating a diff.

Stefan Krah wrote:
> 
> Stefan Krah <stefan-usenet@bytereef.org> added the comment:
> 
> The builds are almost identical, so I attach a diff of the build output.
> For both builds, I used a fresh Python-3.1.1 directory. This looks
> suspicious:
> 
> -checking whether va_list is an array... yes
> +checking whether va_list is an array... no
> 
> 
> For completeness' sake: "-fno-strict-aliasing -m32" were reversed in the
> BASECFLAGS case, so I edited the BASECFLAGS output for the purposes of
> creating a diff.

Thanks.

Looking at the diff I cannot see any difference between the
two builds in terms of gcc options applied during the compile
and link step.

As you noticed, this leaves the va_list difference which also
causes the warnings in the 32-bit build.

I guess this points to the real cause of the problem: configure
doesn't know anything about BASECFLAGS, but does use CFLAGS for
checking various compiler features, so Antoines command line
options using CFLAGS work, while the ones I posted don't.

(1) I can get around the configure problem by patching configure.in,
    meaning that va_list is detected correctly now. Perhaps BASECFLAGS
    should be used by default for the compile tests?

(2) Now I run into the problem that distutils somehow ignores the
    LDFLAGS:

 
gcc -pthread -shared build/temp.linux-x86_64-3.2/home/stefan/tmp/svn/py3k/Modules/_struct.o -L/usr/local/lib -o build/lib.linux-x86_64-3.2/_struct.so
collect2: ld terminated with signal 11 [Segmentation fault]



So it might be possible to fix the situation by changing configure.in
and distutils. On the other hand, the patches from Bob and Antoine are
simpler.

Stefan Krah wrote:
> 
> Stefan Krah <stefan-usenet@bytereef.org> added the comment:
> 
> (1) I can get around the configure problem by patching configure.in,
>     meaning that va_list is detected correctly now. Perhaps BASECFLAGS
>     should be used by default for the compile tests?
> 
> (2) Now I run into the problem that distutils somehow ignores the
>     LDFLAGS:
> 
>  
> gcc -pthread -shared build/temp.linux-x86_64-3.2/home/stefan/tmp/svn/py3k/Modules/_struct.o -L/usr/local/lib -o build/lib.linux-x86_64-3.2/_struct.so
> collect2: ld terminated with signal 11 [Segmentation fault]
> 
> 
> 
> So it might be possible to fix the situation by changing configure.in
> and distutils. On the other hand, the patches from Bob and Antoine are
> simpler.

Right, let's go with Antoine's patch and fix the real Makefile
variable problem in some other release - this will require some
major changes to the Makefile and also some changes to distutils.

Fixed in r79218 (trunk), r79220 (2.6), r79221 (py3k), r79222 (3.1). Thanks!