configure.in uses AC_PROG_CC, extract of the autoconf manual:
(http://www.delorie.com/gnu/docs/autoconf/autoconf_64.html)
 
 If using the GNU C compiler, set shell variable GCC to `yes'. If output variable CFLAGS was not already set, set it to `-g -O2' for the GNU C compiler (`-O2' on systems where GCC does not accept `-g'), or `-g' for other compilers.

Python does already set the optimization level in its OPT variable: for gcc, it uses -O3 by default, and not -O option (in this case, gcc disables all optimisations, it's like -O0) if --with-pydebug is used.

Because of AC_PROG_CC, Python is compiled with -O2 even if --with-pydebug is used, which is bad because it's harder to debug an optimized program: most variable are unavailable (gcc prints "<optimized out>").

Another problem is that AC_PROG_CC eats user CFLAGS. It's not possible to specify: ./configure CFLAGS="myflags".

On the autoconf mailing list, I saw a simple "trick": 

   Save CFLAGS before you call AC_PROG_CC, and restore it after, 
   if you don't want "-g -O2".

Attached patch implements that. Results:
 * ./configure: CFLAGS=$(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
 * ./configure --with-pdebug CFLAGS="-O0": CFLAGS=$(BASECFLAGS) -O0 $(OPT) $(EXTRA_CFLAGS)

It works :-)

Setting CFLAGS is broken in Python configure system, so it's better not to rely on it (or to fix it, but that's a major task - the whole CFLAGS and LDFLAGS system used in Python's configure has over the years turned into a complete mess).

You should get the same result by using ./configure OPT="-O0".

> (or to fix it, but that's a major task - the whole CFLAGS 
> and LDFLAGS system used in Python's configure has over the
> years turned into a complete mess).

What do you mean by "a complete mess"? Did you try my patch? Is it enough to fix this issue?

A least, my patch is enough to disable all compilation optimization if --with-pydebug option is used (which was my first goal).

STINNER Victor wrote:
> 
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
> 
>> (or to fix it, but that's a major task - the whole CFLAGS 
>> and LDFLAGS system used in Python's configure has over the
>> years turned into a complete mess).
> 
> What do you mean by "a complete mess"? 

There's a discussion on some other ticket about this. Basically,
Python's configure script doesn't really allow setting CFLAGS
in a meaningful way via env vars (which it should to be standards
conform).

It adds all sorts of non-standard variables which are then combined
to build the final CFLAGS variable: BASECFLAGS, EXTRA_CFLAGS and
OPT.

> Did you try my patch? Is it enough to fix this issue?

No. It doesn't allow overriding the CFLAGS settings actually
used by the system. You'd also need to set BASECFLAGS="", OPT="" and
EXTRA_CFLAGS="" in addition to applying your patch and setting
CFLAGS to "-O0".

> A least, my patch is enough to disable all compilation optimization if --with-pydebug option is used (which was my first goal).

It unconditionally overrides CFLAGS - even if it is not set and
defined by AC_PROG_CC as "-g -O2". That would need to be corrected.

Other than that it does help a little work around the mess :-)

MaL> It unconditionally overrides CFLAGS - even if it is not 
MaL> set and defined by AC_PROG_CC as "-g -O2". That would need 
MaL> to be corrected.
MaL>
MaL> Other than that it does help a little work around the mess :-)

I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)

On my Linux, .c files are now compiled with:
 - "-fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall ..." (default)
 - "-fno-strict-aliasing -g -Wall ..." (--with-pydebug): no more compiler optimization hurting gdb ;-)

> I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)

The buildbots look happy => r79401 (py3k), blocked in 2.6 and 3.1

The issue title was "configure: ignore AC_PROG_CC hardcoded CFLAGS", and not "fix configure", so I can close the issue.

Marc-Andre: open a new issue if you are motivated to fix "configure mess" :-)

STINNER Victor wrote:
> 
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
> 
> MaL> It unconditionally overrides CFLAGS - even if it is not 
> MaL> set and defined by AC_PROG_CC as "-g -O2". That would need 
> MaL> to be corrected.
> MaL>
> MaL> Other than that it does help a little work around the mess :-)
> 
> I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)
> 
> On my Linux, .c files are now compiled with:
>  - "-fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall ..." (default)
>  - "-fno-strict-aliasing -g -Wall ..." (--with-pydebug): no more compiler optimization hurting gdb ;-)

The patch you checked in still unconditionally overrides the
CFLAGS setting applied by AC_PROG_CC in case no CFLAGS variable
is set.

Please fix that or revert the patch.

STINNER Victor wrote:
> 
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
> 
>> I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)
> 
> The buildbots look happy => r79401 (py3k), blocked in 2.6 and 3.1
> 
> The issue title was "configure: ignore AC_PROG_CC hardcoded CFLAGS", and not "fix configure", so I can close the issue.

The issue now is: AC_PROG_CC no longer initializes CFLAGS if not set.

> Marc-Andre: open a new issue if you are motivated to fix "configure mess" :-)

Some other day...

MaL> The patch you checked in still unconditionally overrides the
MaL> CFLAGS setting applied by AC_PROG_CC in case no CFLAGS variable
MaL> is set.
MaL>
MaL> The issue now is: AC_PROG_CC no longer initializes CFLAGS 
MaL> if not set.

Sorry, but I don't understand. Why should it be initialized?

I don't like the default value because it enables optimization when --with-pydebug is used and I consider that as a bug. If no configure option is used, Python adds -O3 as before. About -g: Python always add it, so the -g from AC_PROG_CC was redundant.

In Makefile.pre.in, prefixes and suffixes are added to the CFLAGS: "CFLAGS= $(BASECFLAGS) @CFLAGS@ $(OPT) $(EXTRA_CFLAGS)". But I consider this as a separate issue.

STINNER Victor wrote:
> 
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
> 
> MaL> The patch you checked in still unconditionally overrides the
> MaL> CFLAGS setting applied by AC_PROG_CC in case no CFLAGS variable
> MaL> is set.
> MaL>
> MaL> The issue now is: AC_PROG_CC no longer initializes CFLAGS 
> MaL> if not set.
> 
> Sorry, but I don't understand. Why should it be initialized?

Sorry, perhaps I wasn't clear enough.

The purpose of AC_PROG_CC is to setup the env vars depending on
the found C compiler. One of these settings is the CFLAGS env
var and you patch causes those settings to be overwritten
if CFLAGS has not been defined as env var before starting
./configure.

Since CFLAGS will most of the time not be set by the user
running ./configure, the patch effectively disables the
default settings determined based on the compiler by AC_PROG_CC.

> I don't like the default value because it enables optimization when --with-pydebug is used and I consider that as a bug. If no configure option is used, Python adds -O3 as before. About -g: Python always add it, so the -g from AC_PROG_CC was redundant.

Right, but it's possible that AC_PROG_CC adds other flags
as well (it currently doesn't, but that may well change in
future versions of autoconf).

I'd suggest to only override the CFLAGS setting in case it
was defined before running the AC_PROG_CC code.

In addition, it may be useful to have --with-pydebug replace OPT
with "-O0" or add "-O0" to it the end of it.

> In Makefile.pre.in, prefixes and suffixes are added to the CFLAGS: "CFLAGS= $(BASECFLAGS) @CFLAGS@ $(OPT) $(EXTRA_CFLAGS)". But I consider this as a separate issue.

Right. That's for historic reasons. OPT is very old, BASECFLAGS
and EXTRA_CFLAGS are newer additions.

Note these changes to restore CFLAGS have the side effect of breaking OS X universal builds; see Issue8366.

Victor, could you please fix the patch or revert it ?

Thanks.

Reopening the ticket: it shouldn't have been closed.

I'm also making this a release blocker, since this needs to be fixed before the next release - the CC variable has to be initialized by the build system and breaking this in general for all default builds just to get a debug build without optimizations is not warranted.

To be totally fair, it is likely that part of the OS X breakage was caused by the original code inadvertently working around the original CFLAGS misbehavior. From an OS X perspective, it may be best to just fix the new issue and move on.

Issue #8366 was caused by a fix of issue #1628484 (and ok, indirectly by my change). Issue #8366 is now fixed. Can I close this issue again or do you think that there is still something to do?

I think the issue can be closed again.

Even if this issue doesn't fix all the configure "complete mess", I think that it improves it a little bit. Open new issues if you would like to fix other parts of the configure script.

> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
> 
> Even if this issue doesn't fix all the configure "complete mess", I think that it improves it a little bit. Open new issues if you would like to fix other parts of the configure script.

Viktor, you are still missing the point and please stop closing the issue again and again.

Please change the configure.in script to only override the CFLAGS in case they were set before entering the AC_PROG_CC part !

I've explained that over and over again and I don't understand why you are being completely ignorant of the implications of your change.

The Mac OS X fix is unrelated to this change.

Marc-Andre Lemburg wrote:
> 
> Marc-Andre Lemburg <mal@egenix.com> added the comment:
> 
> Please change the configure.in script to only override the CFLAGS in case they were set before entering the AC_PROG_CC part !

Sorry, this should have read:

... if CFLAGS *were* set before entering AC_PROG_CC ...

> I've explained that over and over again and I don't understand why you are being completely ignorant of the implications of your change.

One of the main purposes of AC_PROG_CC is to setup CFLAGS in case
they are not and your change defeats this purpose.

Sorry but i don't really understand the problem of my patch, and I don't want to spend time of this. Revert my patch if you think that it introduced a regression.

STINNER Victor wrote:
> 
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
> 
> Sorry but i don't really understand the problem of my patch, and I don't want to spend time of this. Revert my patch if you think that it introduced a regression.

See the new implementation for what I meant...

r80665 - in python/trunk: configure configure.in
r80666 - in python/branches/py3k: configure	configure.in

[issue8211] configure: ignore AC_PROG_CC hardcoded CFLAGS

Only override the AC_PROG_CC determined CFLAGS if they were set by the user.
This restores the default behavior in the common case of not having CFLAGS
defined when running configure.

Since r80665, a

./configure --with-pydebug

seems to give compilation with -O2 (tested on OS X and Linux).  Is this intentional?

I'm getting, e.g.,

gcc -c  -g -O2 -g -Wall -Wstrict-prototypes  -I. -IInclude -I./Include   -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c

Mark Dickinson wrote:
> 
> Mark Dickinson <dickinsm@gmail.com> added the comment:
> 
> Since r80665, a
> 
> ./configure --with-pydebug
> 
> seems to give compilation with -O2 (tested on OS X and Linux).  Is this intentional?

Yes. I've restored the previous behavior of configure to
have AC_PROG_CC determine CFLAGS defaults.

Please open a new ticket to have --with-pydebug disable use
of any optimization flags. We need to find a different
solution for that. Unconditionally ignoring the AC_PROG_CC
CFLAGS defaults is not solution.

Marc-Andre Lemburg wrote:
> 
> Marc-Andre Lemburg <mal@egenix.com> added the comment:
> 
> Mark Dickinson wrote:
>>
>> Mark Dickinson <dickinsm@gmail.com> added the comment:
>>
>> Since r80665, a
>>
>> ./configure --with-pydebug
>>
>> seems to give compilation with -O2 (tested on OS X and Linux).  Is this intentional?
> 
> Yes. I've restored the previous behavior of configure to
> have AC_PROG_CC determine CFLAGS defaults.
> 
> Please open a new ticket to have --with-pydebug disable use
> of any optimization flags. We need to find a different
> solution for that. Unconditionally ignoring the AC_PROG_CC
> CFLAGS defaults is not solution.

Note that using the following line will disable the AC_PROG_CC
defaults:

./configure --with-pydebug CFLAGS=''

This is a new feature that was introduced previously by Victor
and that I corrected in r80665.

> Yes. I've restored the previous behavior of configure to
> have AC_PROG_CC determine CFLAGS defaults.

Just to be clear, the previous behaviour has *not* been restored.  Up until r80665, a '--with-pydebug' build did not include optimization.  Since r80665, it does.

Ah, I understand now: -O2 started appearing in debug builds in r79218, which changed the Makefile to respect CFLAGS.  I tested a variety of revision, but failed to test revision in between that and Victor's change...

Mark Dickinson wrote:
> 
> Mark Dickinson <dickinsm@gmail.com> added the comment:
> 
> Ah, I understand now: -O2 started appearing in debug builds in r79218, which changed the Makefile to respect CFLAGS.  I tested a variety of revision, but failed to test revision in between that and Victor's change...

Right. I was referring to r79391, ie. the state before Victor checked
in his patch.