Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

configure: allow user-provided CFLAGS to override AC_PROG_CC defaults #52458

Closed
vstinner opened this issue Mar 23, 2010 · 27 comments
Closed

configure: allow user-provided CFLAGS to override AC_PROG_CC defaults #52458

vstinner opened this issue Mar 23, 2010 · 27 comments
Assignees
@malemburg
Labels
build The build process and cross-build

Comments

@vstinner
Copy link
Member

BPO 8211
Nosy @malemburg, @loewis, @mdickinson, @vstinner, @ned-deily
Files
  • configure_cflags.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/malemburg'
closed_at = <Date 2010-05-05.12:35:14.217>
created_at = <Date 2010-03-23.12:14:06.227>
labels = ['build']
title = 'configure: allow user-provided CFLAGS to override AC_PROG_CC defaults'
updated_at = <Date 2010-05-05.13:26:51.147>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2010-05-05.13:26:51.147>
actor = 'lemburg'
assignee = 'lemburg'
closed = True
closed_date = <Date 2010-05-05.12:35:14.217>
closer = 'lemburg'
components = ['Build']
creation = <Date 2010-03-23.12:14:06.227>
creator = 'vstinner'
dependencies = []
files = ['16628']
hgrepos = []
issue_num = 8211
keywords = ['patch']
message_count = 27.0
messages = ['101578', '101589', '101592', '101593', '101660', '101664', '101673', '101674', '101680', '101689', '102811', '102862', '102864', '102868', '103952', '103981', '103983', '103989', '103992', '104649', '104653', '105015', '105019', '105020', '105023', '105025', '105026']
nosy_count = 6.0
nosy_names = ['lemburg', 'loewis', 'mark.dickinson', 'vstinner', 'ned.deily', 'Arfrever']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue8211'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']
    @vstinner
    Copy link
    Member Author

    configure.in uses AC_PROG_CC, extract of the autoconf manual:
    (http://www.delorie.com/gnu/docs/autoconf/autoconf_64.html)

    If using the GNU C compiler, set shell variable GCC to yes'. If output variable CFLAGS was not already set, set it to -g -O2' for the GNU C compiler (-O2' on systems where GCC does not accept -g'), or `-g' for other compilers.

    Python does already set the optimization level in its OPT variable: for gcc, it uses -O3 by default, and not -O option (in this case, gcc disables all optimisations, it's like -O0) if --with-pydebug is used.

    Because of AC_PROG_CC, Python is compiled with -O2 even if --with-pydebug is used, which is bad because it's harder to debug an optimized program: most variable are unavailable (gcc prints "<optimized out>").

    Another problem is that AC_PROG_CC eats user CFLAGS. It's not possible to specify: ./configure CFLAGS="myflags".

    On the autoconf mailing list, I saw a simple "trick":

    Save CFLAGS before you call AC_PROG_CC, and restore it after,
    if you don't want "-g -O2".

    Attached patch implements that. Results:

    • ./configure: CFLAGS=$(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
    • ./configure --with-pdebug CFLAGS="-O0": CFLAGS=$(BASECFLAGS) -O0 $(OPT) $(EXTRA_CFLAGS)

    It works :-)

    @vstinner vstinner added the build The build process and cross-build label Mar 23, 2010
    @malemburg
    Copy link
    Member

    Setting CFLAGS is broken in Python configure system, so it's better not to rely on it (or to fix it, but that's a major task - the whole CFLAGS and LDFLAGS system used in Python's configure has over the years turned into a complete mess).

    You should get the same result by using ./configure OPT="-O0".

    @vstinner
    Copy link
    Member Author

    (or to fix it, but that's a major task - the whole CFLAGS
    and LDFLAGS system used in Python's configure has over the
    years turned into a complete mess).

    What do you mean by "a complete mess"? Did you try my patch? Is it enough to fix this issue?

    A least, my patch is enough to disable all compilation optimization if --with-pydebug option is used (which was my first goal).

    @malemburg
    Copy link
    Member

    STINNER Victor wrote:

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    > (or to fix it, but that's a major task - the whole CFLAGS
    > and LDFLAGS system used in Python's configure has over the
    > years turned into a complete mess).

    What do you mean by "a complete mess"?

    There's a discussion on some other ticket about this. Basically,
    Python's configure script doesn't really allow setting CFLAGS
    in a meaningful way via env vars (which it should to be standards
    conform).

    It adds all sorts of non-standard variables which are then combined
    to build the final CFLAGS variable: BASECFLAGS, EXTRA_CFLAGS and
    OPT.

    Did you try my patch? Is it enough to fix this issue?

    No. It doesn't allow overriding the CFLAGS settings actually
    used by the system. You'd also need to set BASECFLAGS="", OPT="" and
    EXTRA_CFLAGS="" in addition to applying your patch and setting
    CFLAGS to "-O0".

    A least, my patch is enough to disable all compilation optimization if --with-pydebug option is used (which was my first goal).

    It unconditionally overrides CFLAGS - even if it is not set and
    defined by AC_PROG_CC as "-g -O2". That would need to be corrected.

    Other than that it does help a little work around the mess :-)

    @vstinner
    Copy link
    Member Author

    MaL> It unconditionally overrides CFLAGS - even if it is not
    MaL> set and defined by AC_PROG_CC as "-g -O2". That would need
    MaL> to be corrected.
    MaL>
    MaL> Other than that it does help a little work around the mess :-)

    I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)

    On my Linux, .c files are now compiled with:

    • "-fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall ..." (default)
    • "-fno-strict-aliasing -g -Wall ..." (--with-pydebug): no more compiler optimization hurting gdb ;-)

    @vstinner
    Copy link
    Member Author

    I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)

    The buildbots look happy => r79401 (py3k), blocked in 2.6 and 3.1

    The issue title was "configure: ignore AC_PROG_CC hardcoded CFLAGS", and not "fix configure", so I can close the issue.

    Marc-Andre: open a new issue if you are motivated to fix "configure mess" :-)

    @malemburg
    Copy link
    Member

    STINNER Victor wrote:

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    MaL> It unconditionally overrides CFLAGS - even if it is not
    MaL> set and defined by AC_PROG_CC as "-g -O2". That would need
    MaL> to be corrected.
    MaL>
    MaL> Other than that it does help a little work around the mess :-)

    I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)

    On my Linux, .c files are now compiled with:

    • "-fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall ..." (default)
    • "-fno-strict-aliasing -g -Wall ..." (--with-pydebug): no more compiler optimization hurting gdb ;-)

    The patch you checked in still unconditionally overrides the
    CFLAGS setting applied by AC_PROG_CC in case no CFLAGS variable
    is set.

    Please fix that or revert the patch.

    @malemburg
    Copy link
    Member

    STINNER Victor wrote:

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    > I commited my patch: r79392 (trunk). I'm waiting for the buildbots before porting to other branches :-)

    The buildbots look happy => r79401 (py3k), blocked in 2.6 and 3.1

    The issue title was "configure: ignore AC_PROG_CC hardcoded CFLAGS", and not "fix configure", so I can close the issue.

    The issue now is: AC_PROG_CC no longer initializes CFLAGS if not set.

    Marc-Andre: open a new issue if you are motivated to fix "configure mess" :-)

    Some other day...

    @vstinner
    Copy link
    Member Author

    MaL> The patch you checked in still unconditionally overrides the
    MaL> CFLAGS setting applied by AC_PROG_CC in case no CFLAGS variable
    MaL> is set.
    MaL>
    MaL> The issue now is: AC_PROG_CC no longer initializes CFLAGS
    MaL> if not set.

    Sorry, but I don't understand. Why should it be initialized?

    I don't like the default value because it enables optimization when --with-pydebug is used and I consider that as a bug. If no configure option is used, Python adds -O3 as before. About -g: Python always add it, so the -g from AC_PROG_CC was redundant.

    In Makefile.pre.in, prefixes and suffixes are added to the CFLAGS: "CFLAGS= $(BASECFLAGS) @CFLAGS@ $(OPT) $(EXTRA_CFLAGS)". But I consider this as a separate issue.

    @malemburg
    Copy link
    Member

    STINNER Victor wrote:

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    MaL> The patch you checked in still unconditionally overrides the
    MaL> CFLAGS setting applied by AC_PROG_CC in case no CFLAGS variable
    MaL> is set.
    MaL>
    MaL> The issue now is: AC_PROG_CC no longer initializes CFLAGS
    MaL> if not set.

    Sorry, but I don't understand. Why should it be initialized?

    Sorry, perhaps I wasn't clear enough.

    The purpose of AC_PROG_CC is to setup the env vars depending on
    the found C compiler. One of these settings is the CFLAGS env
    var and you patch causes those settings to be overwritten
    if CFLAGS has not been defined as env var before starting
    ./configure.

    Since CFLAGS will most of the time not be set by the user
    running ./configure, the patch effectively disables the
    default settings determined based on the compiler by AC_PROG_CC.

    I don't like the default value because it enables optimization when --with-pydebug is used and I consider that as a bug. If no configure option is used, Python adds -O3 as before. About -g: Python always add it, so the -g from AC_PROG_CC was redundant.

    Right, but it's possible that AC_PROG_CC adds other flags
    as well (it currently doesn't, but that may well change in
    future versions of autoconf).

    I'd suggest to only override the CFLAGS setting in case it
    was defined before running the AC_PROG_CC code.

    In addition, it may be useful to have --with-pydebug replace OPT
    with "-O0" or add "-O0" to it the end of it.

    In Makefile.pre.in, prefixes and suffixes are added to the CFLAGS: "CFLAGS= $(BASECFLAGS) @CFLAGS@ $(OPT) $(EXTRA_CFLAGS)". But I consider this as a separate issue.

    Right. That's for historic reasons. OPT is very old, BASECFLAGS
    and EXTRA_CFLAGS are newer additions.

    @ned-deily
    Copy link
    Member

    Note these changes to restore CFLAGS have the side effect of breaking OS X universal builds; see bpo-8366.

    @malemburg
    Copy link
    Member

    Victor, could you please fix the patch or revert it ?

    Thanks.

    @malemburg
    Copy link
    Member

    Reopening the ticket: it shouldn't have been closed.

    I'm also making this a release blocker, since this needs to be fixed before the next release - the CC variable has to be initialized by the build system and breaking this in general for all default builds just to get a debug build without optimizations is not warranted.

    @malemburg malemburg reopened this Apr 11, 2010
    @ned-deily
    Copy link
    Member

    To be totally fair, it is likely that part of the OS X breakage was caused by the original code inadvertently working around the original CFLAGS misbehavior. From an OS X perspective, it may be best to just fix the new issue and move on.

    @vstinner
    Copy link
    Member Author

    Issue bpo-8366 was caused by a fix of issue bpo-1628484 (and ok, indirectly by my change). Issue bpo-8366 is now fixed. Can I close this issue again or do you think that there is still something to do?

    @ned-deily
    Copy link
    Member

    I think the issue can be closed again.

    @vstinner
    Copy link
    Member Author

    Even if this issue doesn't fix all the configure "complete mess", I think that it improves it a little bit. Open new issues if you would like to fix other parts of the configure script.

    @malemburg
    Copy link
    Member

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    Even if this issue doesn't fix all the configure "complete mess", I think that it improves it a little bit. Open new issues if you would like to fix other parts of the configure script.

    Viktor, you are still missing the point and please stop closing the issue again and again.

    Please change the configure.in script to only override the CFLAGS in case they were set before entering the AC_PROG_CC part !

    I've explained that over and over again and I don't understand why you are being completely ignorant of the implications of your change.

    The Mac OS X fix is unrelated to this change.

    @malemburg malemburg reopened this Apr 23, 2010
    @malemburg
    Copy link
    Member

    Marc-Andre Lemburg wrote:

    Marc-Andre Lemburg <mal@egenix.com> added the comment:

    Please change the configure.in script to only override the CFLAGS in case they were set before entering the AC_PROG_CC part !

    Sorry, this should have read:

    ... if CFLAGS *were* set before entering AC_PROG_CC ...

    I've explained that over and over again and I don't understand why you are being completely ignorant of the implications of your change.

    One of the main purposes of AC_PROG_CC is to setup CFLAGS in case
    they are not and your change defeats this purpose.

    @vstinner
    Copy link
    Member Author

    Sorry but i don't really understand the problem of my patch, and I don't want to spend time of this. Revert my patch if you think that it introduced a regression.

    @malemburg
    Copy link
    Member

    STINNER Victor wrote:

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    Sorry but i don't really understand the problem of my patch, and I don't want to spend time of this. Revert my patch if you think that it introduced a regression.

    See the new implementation for what I meant...

    r80665 - in python/trunk: configure configure.in
    r80666 - in python/branches/py3k: configure configure.in

    [bpo-8211] configure: ignore AC_PROG_CC hardcoded CFLAGS

    Only override the AC_PROG_CC determined CFLAGS if they were set by the user.
    This restores the default behavior in the common case of not having CFLAGS
    defined when running configure.

    @malemburg malemburg changed the title configure: ignore AC_PROG_CC hardcoded CFLAGS configure: allow user-provided CFLAGS to override AC_PROG_CC defaults Apr 30, 2010
    @malemburg malemburg self-assigned this Apr 30, 2010
    @mdickinson
    Copy link
    Member

    Since r80665, a

    ./configure --with-pydebug

    seems to give compilation with -O2 (tested on OS X and Linux). Is this intentional?

    I'm getting, e.g.,

    gcc -c -g -O2 -g -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c

    @mdickinson mdickinson reopened this May 5, 2010
    @malemburg
    Copy link
    Member

    Mark Dickinson wrote:

    Mark Dickinson <dickinsm@gmail.com> added the comment:

    Since r80665, a

    ./configure --with-pydebug

    seems to give compilation with -O2 (tested on OS X and Linux). Is this intentional?

    Yes. I've restored the previous behavior of configure to
    have AC_PROG_CC determine CFLAGS defaults.

    Please open a new ticket to have --with-pydebug disable use
    of any optimization flags. We need to find a different
    solution for that. Unconditionally ignoring the AC_PROG_CC
    CFLAGS defaults is not solution.

    @malemburg
    Copy link
    Member

    Marc-Andre Lemburg wrote:

    Marc-Andre Lemburg <mal@egenix.com> added the comment:

    Mark Dickinson wrote:
    >
    > Mark Dickinson <dickinsm@gmail.com> added the comment:
    >
    > Since r80665, a
    >
    > ./configure --with-pydebug
    >
    > seems to give compilation with -O2 (tested on OS X and Linux). Is this intentional?

    Yes. I've restored the previous behavior of configure to
    have AC_PROG_CC determine CFLAGS defaults.

    Please open a new ticket to have --with-pydebug disable use
    of any optimization flags. We need to find a different
    solution for that. Unconditionally ignoring the AC_PROG_CC
    CFLAGS defaults is not solution.

    Note that using the following line will disable the AC_PROG_CC
    defaults:

    ./configure --with-pydebug CFLAGS=''

    This is a new feature that was introduced previously by Victor
    and that I corrected in r80665.

    @mdickinson
    Copy link
    Member

    Yes. I've restored the previous behavior of configure to
    have AC_PROG_CC determine CFLAGS defaults.

    Just to be clear, the previous behaviour has *not* been restored. Up until r80665, a '--with-pydebug' build did not include optimization. Since r80665, it does.

    @mdickinson
    Copy link
    Member

    Ah, I understand now: -O2 started appearing in debug builds in r79218, which changed the Makefile to respect CFLAGS. I tested a variety of revision, but failed to test revision in between that and Victor's change...

    @malemburg
    Copy link
    Member

    Mark Dickinson wrote:

    Mark Dickinson <dickinsm@gmail.com> added the comment:

    Ah, I understand now: -O2 started appearing in debug builds in r79218, which changed the Makefile to respect CFLAGS. I tested a variety of revision, but failed to test revision in between that and Victor's change...

    Right. I was referring to r79391, ie. the state before Victor checked
    in his patch.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @malemburg malemburg

    Labels
    build The build process and cross-build
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @vstinner @mdickinson @ned-deily @malemburg