Problem description: Trying to cross-compile $(LIBRARY) (libpython2.7.a for example) causes "pgen" to be built, even when it's not used in the cross-compilation case (only a file copy is done to generate $(GRAMMAR_H) and $(GRAMMAR_C)).


The current rule for $(PGEN) in Makefile.pre.in does not include $(CFLAGS):

https://hg.python.org/cpython/file/tip/Makefile.pre.in#l810

This causes problems when $(CFLAGS) changes the ARM float ABI, e.g.:

CFLAGS="-mfloat-abi=hard"

This causes the following issues at link time:

 1. The .o files that get linked into "pgen" are built with CFLAGS
    (which is good, because some of them are used for libpython as well)
 2. When the "pgen" binary gets built, the $(CFLAGS) are not used
 3. Compiler fails to build "pgen" with different float ABI settings

=====

[...]
arm-none-eabi-gcc -c -fno-strict-aliasing -march=armv6k -mtune=mpcore -mfloat-abi=hard -mtp=soft -fomit-frame-pointer -ffunction-sections -DARM11 -D_3DS -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include   -DPy_BUILD_CORE -o Parser/pgenmain.o Parser/pgenmain.c
arm-none-eabi-gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Objects/obmalloc.o Python/mysnprintf.o Python/pyctype.o Parser/tokenizer_pgen.o Parser/printgrammar.o Parser/pgenmain.o  -o Parser/pgen
/Users/thp/pkg/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/5.3.0/../../../../arm-none-eabi/bin/ld: error: Parser/acceler.o uses VFP register arguments, Parser/pgen does not
/Users/thp/pkg/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/5.3.0/../../../../arm-none-eabi/bin/ld: failed to merge target specific data of file Parser/acceler.o
[...]

=====

Note that the error message is repeated for all .o files linked into pgen, I've only included one here for demonstration purposes. The following patch (against a Python 2.7.12 tarball, similar fix for Hg tip and Python 3) fixes the issue for me:

=====
diff -u Python-2.7.12/Makefile.pre.in Python-2.7.12-fix/Makefile.pre.in
--- Python-2.7.12/Makefile.pre.in	2016-06-25 23:49:31.000000000 +0200
+++ Python-2.7.12-fix/Makefile.pre.in	2016-07-12 00:17:02.000000000 +0200
@@ -698,7 +698,7 @@
 	fi
 
 $(PGEN):	$(PGENOBJS)
-		$(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN)
+		$(CC) $(OPT) $(CFLAGS) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN)
 
 Parser/grammar.o:	$(srcdir)/Parser/grammar.c \
 				$(srcdir)/Include/token.h \
=====

Also note that the same $(CFLAGS) needs to be added to the rule for $(BUILDPYTHON) if one wants to build that as well, but in my case, I only did a "make libpython2.7.a", and that indirectly depends on pgen ($(LIBRARY) -> $(LIBRARY_OBJS) -> $(PYTHON_OBJS) -> Python/graminit.o -> $(GRAMMAR_C) -> $(GRAMMAR_H) -> $(PGEN), which results in that error message, so libpython2.7.a can't be built).

Another fix could be to make it so that $(GRAMMAR_H) does not depend on $(PGEN) if $(cross_compiling) is "yes" (if you read the rule contents for $(GRAMMAR_H), you'll find that indeed $(PGEN) isn't used at all if $(cross_compiling) is "yes". At least for GNU make, it might be possible to avoid building "pgen" in that case as follows and removing $(PGEN) from the default dependencies of $(GRAMMAR_H):

ifneq ($(cross_compiling),yes)
$(GRAMMAR_H): $(PGEN)
endif

If this is a more acceptable solution, one could probably rewrite the "test "$(cross_compiling" != "yes"; then..." part of the make rules from $(GRAMMAR_H) and $(GRAMMAR_C) with Make's ifeq, here's a patch for that instead (this also makes the dependencies more clear, since $(GRAMMAR_H) does not depend on $(GRAMMAR_INPUT) for the cross-input case, as it is not used):

=====
diff -u Python-2.7.12/Makefile.pre.in Python-2.7.12-fix/Makefile.pre.in
--- Python-2.7.12/Makefile.pre.in	2016-06-25 23:49:31.000000000 +0200
+++ Python-2.7.12-fix/Makefile.pre.in	2016-07-12 00:37:43.000000000 +0200
@@ -680,22 +680,21 @@
 
 Modules/pwdmodule.o: $(srcdir)/Modules/pwdmodule.c $(srcdir)/Modules/posixmodule.h
 
+ifeq ($(cross_compiling),yes)
+$(GRAMMAR_H): $(srcdir)/Include/graminit.h
+	@$(MKDIR_P) Include
+	cp $(srcdir)/Include/graminit.h $(GRAMMAR_H).tmp
+	mv $(GRAMMAR_H).tmp $(GRAMMAR_H)
+$(GRAMMAR_C): $(srcdir)/Python/graminit.c
+	cp $(srcdir)/Python/graminit.c $(GRAMMAR_C).tmp
+	mv $(GRAMMAR_C).tmp $(GRAMMAR_C)
+else
 $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN)
 	@$(MKDIR_P) Include
-	# Avoid copying the file onto itself for an in-tree build
-	if test "$(cross_compiling)" != "yes"; then \
-		$(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C); \
-	else \
-		cp $(srcdir)/Include/graminit.h $(GRAMMAR_H).tmp; \
-		mv $(GRAMMAR_H).tmp $(GRAMMAR_H); \
-	fi
+	$(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
 $(GRAMMAR_C): $(GRAMMAR_H)
-	if test "$(cross_compiling)" != "yes"; then \
-		touch $(GRAMMAR_C); \
-	else \
-		cp $(srcdir)/Python/graminit.c $(GRAMMAR_C).tmp; \
-		mv $(GRAMMAR_C).tmp $(GRAMMAR_C); \
-	fi
+	touch $(GRAMMAR_C)
+endif
 
 $(PGEN):	$(PGENOBJS)
 		$(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN)
=====

See also (slightly related, the "avoid running" part is fixed, but the "avoid building" part is not fixed): https://bugs.python.org/issue22625