Hmm things are quite complicated. Brief: the build is broken for out-of-source cross-compiling if the host Python is an in-source build.

(Below $build_dir refers to the directory that invokes $source_dir/configure)

In an out-of-source build, setup.py relies on sysconfig.get_config_var('srcdir') to get correct filenames. In _init_posix(), _get_sysconfigdata_name() still returns an incorrect name (for example, _sysconfigdata_m_linux_x86_64-linux-gnu, on Linux x86_64). There's no _sysconfigdata_m_linux_x86_64-linux-gnu.py in $build_dir/build/lib.linux-aarch64-3.6, so $PYTHON_FOR_BUILD imports the one from the host python, which has srcdir == '.' As a result, build_ext can't find source files:

building 'xxlimited' extension
/home/yen/Projects/python3-android/clang-bin/cc -fPIC -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -g -fvar-tracking-assignments -g -fvar-tracking-assignments -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -g -fvar-tracking-assignments -g -fvar-tracking-assignments -std=c99 -DPy_LIMITED_API=0x03050000 -I./Include -I. -IInclude -I/usr/include -I/usr/local/include -I/home/yen/Projects/python3-android/src/cpython/build-target/Include -I/home/yen/Projects/python3-android/src/cpython/build-target -c xxlimited.c -o build/temp.linux-aarch64-3.6/xxlimited.o
clang: error: unknown argument: '-fvar-tracking-assignments'
clang: error: unknown argument: '-fvar-tracking-assignments'
clang: error: unknown argument: '-fvar-tracking-assignments'
clang: error: unknown argument: '-fvar-tracking-assignments'
clang: error: no such file or directory: 'xxlimited.c'
clang: error: no input files

Ironically, before 1d71ce4531ac, $build_dir/build/lib.linux-aarch64-3.6 has _sysconfigdata_m_linux_x86_64-linux-gnu.py, so building is OK.

Again, I don't know to fix it :(

[1] https://hg.python.org/cpython/file/1d71ce4531ac/setup.py#l218