New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BASECFLAGS are not passed to module build line #40367
Comments
The value of BASECFLAGS from This is insufficient when BASECFLAGS contains I did try to set CFLAGS in my environment, as directed |
Logged In: YES This is still a bug in Python 2.4.2. |
Logged In: YES I don't think I will do anything about this anytime soon, so |
I'm seeing a variation of this bug in python 2.5. As far as I can tell in python 2.4.3 on linux it passes BASECFLAGS and OPT, appending CFLAGS from the environment to that if set. In python 2.5 it passes CFLAGS from the Makefile (which is defined as I think it would be preferable to prepend BASECFLAGS instead of OPT if CFLAGS is set in the environment. On my linux machine after building python 2.5 with CFLAGS set to "-O2 -march=athlon-xp" the Makefile has: OPT= -DNDEBUG -g -O3 -Wall -Wstrict-prototypes If I run a setup.py with CFLAGS unset it runs: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC ... Which is reasonable. If I run it with CFLAGS="-O2 -march=athlon-xp": gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -O2 -march=athlon-xp -fPIC ... Which misses -fno-strict-aliasing and still includes all the general flags that I'm trying to set through CFLAGS. If it used BASECFLAGS from the Makefile instead of OPT it would be: gcc -pthread -fno-strict-aliasing -O2 -march=athlon-xp -fPIC ... Which is what I think is the desired result here. |
I am running into a problem related to this. I am attempting to cross It seems that forcing OPT to stay the same is a disservice generally 2.6 and 3.1 say: There is no ability to override opt. I'd be willing to put together a I am also open to other suggestions about how to get around this. I |
I am attaching a preliminary patch to allow override of $(OPT). I am not sure this is sufficient, but am wary of breaking packages that depend on the existing behaviour. The logic indeed seems wrong, but maybe this is something that has to go in distutils2 rather than distutils. |
Pythons in Debian seem to be immune to this problem, thanks to this patch: http://deb.li/3ku1g (via http://lists.debian.org/debian-python/2010/12/msg00005.html). I haven’t had time to learn the intricacies of make variables yet, so I can’t approve a patch. I’m adding people from the nosy lists of other make variables bugs, hopefully someone will be able to review. |
Duplicate report from Stephan Krah: When CFLAGS are set, distutils drops -fno-strict-aliasing (among other $ python2.7 setup.py build
...
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./src -I/usr/local/include -I/usr/local/include/python2.7 -c src/gmpy.c -o build/temp.linux-x86_64-2.7/src/gmpy.o
...
$ CFLAGS="-fomit-frame-pointer" python2.7 setup.py build
...
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fomit-frame-pointer -fPIC -I./src -I/usr/local/include -I/usr/local/include/python2.7 -c src/gmpy.c -o build/temp.linux-x86_64-2.7/src/gmpy.o
src/gmpy.c: In function ‘_cmp_to_object’:
src/gmpy.c:4692: warning: dereferencing type-punned pointer will break strict-aliasing rules
... I'm not sure if this is intentional. The documentation says: "Compiler flags can also be supplied through setting the CFLAGS To me, this sounds as if they should be appended to the regular flags. |
Éric, the Debian patch looks good to me and it solves my build problem. The only question I have is why EXTRA_CFLAGS still go behind CFLAGS But as it is, the patch is an improvement. I'm attaching the version |
Why is OPT duplicated in get_config_vars(...)? |
Antoine Pitrou <report@bugs.python.org> wrote:
I missed that, thanks.
I think it would go too far to append in three places. If the environment
I don't know. Ideally the Debian people would comment if they had any |
EXTRA_CFLAGS were removed in r38848. Is it necessary to configure Chris, would the new minimal patch solve your problem, too? |
I am not convinced that the minimal patch would work for my original issue. I wanted to be able to override the -march option which shows up in OPT on Fedora. I was cross-compiling to a target architecture that does not support the -march option so I would not be able to provide a different one as override. The proposed minimal patch would leave the OPT value and make it unchangeable because CFLAGS would pull out a value for OPT from the Makefile which shows as in my current Ubuntu system: OPT= -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes The debian-sysconfig-flags looks the most correct because it allows override at any point by emulating the CFLAGS = It still looks like it needs an override for EXTRA_CFLAGS and the logic for LD_SHARED looks wrong. In the Makefile LDSHARED is just: Why does LDSHARED need CFLAGS and CPPFLAGS? When provided as an override but not in the Makefile? Do we need to allow for this override in the case of overriding OPT, BASEFLAGS or EXTRA_CFLAGS? |
I don't understand how you can cross-compile using the host Python
EXTRA_CFLAGS is not defined in the Makefile so it would probably be |
Chris Lambacher <report@bugs.python.org> wrote:
Just out of curiosity: You are cross-compiling a C-extension? How can
I think Antoine is correct: EXTRA_CFLAGS show up as an empty string in As an aside, why do EXTRA_CFLAGS still exist in py3k when CFLAGS I agree that the ldshared logic looks wrong. It is possible to specify However, I don't see how the Debian patch could break anything, unless Any interest in a cleaned up version of the Debian patch (remove I'm asking, since I'm unsure about the degree of frozenness of distutils. |
Antoine said:
The get_config_vars() function parses the host Makefile to get the values that we are discussing overriding.
You are correct, EXTRA_CFLAGS is not in the Makefile, but does provide us with an expected order of evaluation. i.e. Distutils is generating the equivalent command lines from the pieces and we should be respecting the Makefile order for consistency. Also, I don't know if there is a way to insert EXTRA_CFLAGS as an option to the ./configure script when Python is built such that it shows up in the resulting Makefile. If there is no way to get EXTRA_CFLAGS into the Makefile, then we should take it entirely out of the equation in the construction of the cflags var. Stephan Said:
In order to build a C extension, you pretty much have to do it through Distutils and therefore the host python interpreter or else do a lot of extra work to get the right settings for each individual extension. |
I think it would be nice, but it should have a test if possible. |
Since I just noticed this and haven't seen it mentioned already: for the record, the Python Makefile for current versions is affected by this issue. The "sharedmods" target, which calls setup.py to build the standard library shared modules, explicitly passes into Distutils via shell variables values for CC, LDSHARED, and OPT. Unfortunately, as noted in this issue, Distutils does not look for an OPT variable. So, while CC and LDSHARED can be overridden from the make command with either macro arguments or env vars, OPT cannot: only the value determined at configure time will be used. I think that Chris's original distutils_opt_flag.patch should be applied to allow OPT to be overridden, without changing any other current behavior. AFAICT, the only compatibility issue would be if a script happened to already have an OPT env variable defined which would now get used by Distutils. I think the risks of that are pretty small and, in case, much smaller than the more extensive tweaking of Distutils behavior as is done in the Debian patches. My interest in this comes from discovering that the OS X installer build script has been overriding OPT for its own purposes, thereby inadvertently dropping compiler options determined by configure (things like -fwrapv) which can result in a broken interpreter. I've changed the installer build to no longer do that. But that does mean that users of the OS X installer will now see those missing compiler options during extension module builds and it is conceivable that could cause problems for some ext modules and there wouldn't be a simple way to work around them (e.g. by setting an OPT env value). If no one has any strong objections, I'll plan to commit that patch soon. |
OPT should not be used in Distutils at all. Lib/distutils/sysconfig.py should have: Makefile.pre.in should have: PY_CFLAGS is defined as: So then OPT could be overriden when calling See my patch for Distutils in bug bpo-1222585. That patch also cleans handling of flags. |
Arfrever, your proposal is certainly one of many possible solutions and one that would be appropriate if we were designing the Python configure, Makefile, and Distutils components from scratch or looking at major changes to Distutils. But we're not at this point. We all know that Distutils is brittle and I think at this point in its life it is best to change as little as possible without really good reason. For better or worse, most distributors and package maintainers have figured out how to make Distutils do what they need to do until the next generation of package building starts being addressed post-3.4. The proposed patch solves a very specific problem with very little risk (IMO) to breaking anyone's current solutions. |
This issue has been fixed by bpo-36235. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: