My build machine is a Haswell Intel x86-64. I'm cross-compiling to x86-64, with -mtune=Skylake -avx2.  During make install PYTHON_FOR_BUILD loads modules from the *build* Lib/ which contain instructions my Haswell can't execute:

| _PYTHON_PROJECT_BASE=/data/poky-tmp/master/work/corei7-64-poky-linux/python3/3.7.2-r0/build _PYTHON_HOST_PLATFORM=linux-x86_64 PYTHONPATH=../Python-3.7.2/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_m_linux_x86_64-linux-gnu python3.7 -v -S -m sysconfig --generate-posix-vars ;\
Illegal instruction

From what I can tell:

configure.ac sets PYTHON_FOR_BUILD like this if cross-compiling:

PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH) '$interp

Note how PYTHONPATH is set to the source and build paths for Lib/.

The intention appears to be that the sysconfig.py in the build is used.  In my case that directory is also full of shared libraries that Python happily loads, and then fails.

Thanks for the report. Unfortunately, as far as I know, we don't claim anywhere to support cross-compiling in general.  As you've probably noticed, the Python Makefile-based build system (for non-Windows platforms) is by no means a conventional GNU autotools one. Over the years there have been some efforts to add support for a handful of specific cross-compilation scenarios, for example compiling for Android.  But I'm afraid you are more or less on your own for the case here.  That said, if you or someone else is motivated to supply some fixes to make more cross-compiling cases work, feel free to reopen the issue and submit the fixes as PRs and they may be considered.  What would probably be the best would be to try to reimplement our build system from the ground up with modern Autotools or something similar but that's a *really* big deal.

@Ross

> The intention appears to be that the sysconfig.py in the build is used.  In my case that directory is also full of shared libraries that Python happily loads, and then fails.

No, the intention is that the native python finds the sysconfigdata module named $_PYTHON_SYSCONFIGDATA_NAME that is located in the directory where the extensions modules (.so libraries) have been cross-compiled. $_PYTHON_SYSCONFIGDATA_NAME is a pure python module, so it can be loaded by the native python.

Please note that in your first post PYTHONPATH is missing the first part, i.e. the path to this directory obtained with:
    $(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)

When PYTHON_FOR_BUILD runs, it does not load any extension module so the Illegal instruction may be caused by some other problem.

strace disagrees.  By putting strace in PYTHON_FOR_BUILD and then invoking make sharedmods:

| openat(AT_FDCWD, "/data/poky-tmp/master/work/corei7-64-poky-linux/python3/3.7.2-r0/build/build/lib.linux-x86_64-3.7/_heapq.cpython-37m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
| read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\20\0\0\0\0\0\0"..., 832) = 832
| fstat(3, {st_mode=S_IFREG|0755, st_size=62104, ...}) = 0
| mmap(NULL, 23880, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f93d7cb9000
| mmap(0x7f93d7cba000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f93d7cba000
| mmap(0x7f93d7cbb000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f93d7cbb000
| mmap(0x7f93d7cbc000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f93d7cbc000
| close(3)                                = 0
| mprotect(0x7f93d7cbc000, 4096, PROT_READ) = 0
| --- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x7f93d7cbab10} ---
| +++ killed by SIGILL (core dumped) +++
| Illegal instruction
| Makefile:625: recipe for target 'sharedmods' failed

We do have patches but I don't think they affect this part of the build.

The cross-compilation rely on the fact that the cross-compiled shared libraries names are constructed using the configure `--host=HOST-TYPE' command line parameter and therefore cannot be loaded by the native compiler that uses a different suffix for those names: in a cross compilation the 'build' machine is identified with a different name than the 'host' machine or it is not a cross-compilation.

Maybe there is a glitch in this assumption ?

That's exactly the glitch.  I'm cross-compiling to a more powerful IA process from IA.  This *is* a cross-compilation but the triple is the same.  Assuming that you can rely on the loader to not open target binaries when they're on the path to load from is unwise.

Re-opening and setting the stage as 'needs patch': at least the cross-compilation should fail with a clear message when the cross-built libraries names and those of the native interpreter have the same suffix.

> Assuming that you can rely on the loader to not open target binaries when they're on the path to load from is unwise.

Meanwhile a workaround may be to build the native interpreter with Py_DEBUG defined (using the --with-pydebug configure option) so that libraries names become different.