This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add a build infrastructure for Android
Type: enhancement Stage: resolved
Components: Cross-Build Versions: Python 3.7
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: Alex.Willmer, martin.panter, ncoghlan, vstinner, xdegaye, zach.ware
Priority: normal Keywords: patch

Created on 2017-05-17 13:57 by xdegaye, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue29619.patch xdegaye, 2017-05-17 14:23
android-api-24.patch xdegaye, 2017-05-17 14:23
Pull Requests
URL Status Linked Edit
PR 1629 closed xdegaye, 2017-05-17 14:17
Messages (8)
msg293852 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-05-17 13:57
This PR adds a build infrastructure for Android on linux that allows a developer to check that source code changes do not break the existing state of this support by testing the changes on Android with a simple command. For example after a change to the socket module:
    $ /path/to/python_src/Android/makesetup && make python PYTHON_ARGS="-m test test_socket"

The README in the PR describes how to install the Android NDK and SDK, how to cross-compile Python, how to run python interactively or to run the Python test suite on the emulator, how to debug python on the emulator with gdb.

This build system has been tested with android-ndk-r13 and android-ndk-r14 which is the latest stable release of the NDK.  UnifiedHeaders [1] introduced in android-ndk-r14 is not yet used by this version of the build system.

[1] https://android.googlesource.com/platform/ndk.git/+/master/docs/UnifiedHeaders.md
msg293856 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-05-17 14:23
The build system for the x86_64, x86 and armv7 architectures has been tested on API 21 and for the x86_64 and arm64 architectures on API 24.  Of these combinations of architecture/api, only x86_64 on API 21 builds correctly today, the others were building correctly few weeks ago but recent changes in Python seem to have broken those builds.

To fix this temporarily:
* Build the 32 bits architectures (x86 and armv7) with the attached issue29619.patch. This is a hack, the problem will be fixed in issue 29619.
* Build on API 24 with the attached android-api-24.patch. This is another hack, I will enter a new issue for those problems which are of a kind that is recurrent with the NDK: a function implemented in libc and whose declaration is missing in the NDK headers, a problem that is supposed to be fixed by UnifiedHeaders.
msg293956 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-05-19 13:21
Commited 'bpo-30386: Support the new NDK Unified Headers'.

This does fix the missing declarations in the NDK headers and android-api-24.patch is not needed now.
It also fixes the missing declaration of sethostname() at API 24 that caused the import of the _socket module to fail.
More surprisingly it also fixes the compilation failure with 32 bits architectures (the size of off_t is now 8 instead of 4 previously) and issue29619.patch is not needed now.
Only one version of the NDK is supported, currently it is android-ndk-r14 the latest stable release.
The '-Wno-nullability-completeness' compilation flag is needed to avoid a flow of warnings in the processing of some NDK headers.

Unified Headers comes with a new annoying problem: langinfo.h exists now but nl_langinfo() is intentionaly not declared except for the __ANDROID_API_FUTURE__ API level
which is reserved for the development of the NDK itself. The solution chosen here is:
* Undefine HAVE_LANGINFO_H and CODESET in Include/pyport.h to maintain the status quo in the Python code.
* The Makefile generated by makesetup runs a script at configure time that checks that nl_langinfo() is not declared and aborts the build otherwise.  This allows to detect when langinfo.h starts being not broken anymore when swicthing to a new version of the NDK. In that case the following steps should be taken upon swicthing to this new version:
  + Remove this script from the build system (obviously).
  + Remove the changes in Include/pyport.h.
  + Revert the changes made in issue 28596.
msg294610 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-05-27 19:57
Details of commit "Add Makefile targets to implement the buildbot steps" in 543a287c32a714a668e426ceb80f4add92cc5de4:

The following buildbot steps, as defined by the UnixBuild class of the master.cfg buildbot configuration file, have now their corresponding commands:
* Configure: makesetup configureFlags && make configure
* Compile:   make host
* Test:      make buildbottest
* Clean:     make hostclean

The 'configure' Makefile target builds the native interpreter, the external libraries and runs configure for the target host on its first execution. Afterwards it mainly runs configure.  As it is, the native interpreter is not distclean(ed) by the above four steps and therefore changes made in Setup.dist are ignored by the successive rebuilds of the native interpreter.
The 'buildbottest' Makefile target invokes the 'buildbottest' target of the host Python Makefile and runs using also the variables: TESTPYTHONOPTS, TESTTIMEOUT, TESTOPTS.

The following changes have also been made:
* The new Makefile target 'kill_emulator' allows to shutdown the emulator.
* All the shell and python scripts are now copied to the emulator at <sys.exec_prefix>/bin (i.e. /data/local/tmp/python/bin).
* The python_shell.sh script run by the user in interactive sessions ('install', 'emulator', 'adb_shell') and the shell scripts run by the 'python' and 'buildbottest' targets now change the current directory to <sys.exec_prefix> (i.e. /data/local/tmp/python) instead of its parent previously.  As a result, the temporary files and directories left over by 'buildbottest' are removed on the next run with the removal of the content of the previous <sys.exec_prefix> directory before the new installation.
* Fix file names whose size exceed MAX_FNAME_SIZE.
msg294776 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-05-30 18:29
Details of commit "Update the native interpreter build on Setup.dist and version changes." in 408c05d44c6310b4a2f0659333560eca6071c8a5:

Native interpreter build:
* Copy Setup.dist to Setup when Setup.dist has been changed.
* Run 'make distclean' upon a Python version change.
Also remove the Misc/NEWS entry until the final commit to avoid merge conflicts.
msg294777 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-05-30 18:46
I don't intend to change the current code in PR 1629 until there is a review or comments asking for changes. Ned Deily has self-requested a review, reviews from other people are also welcome.
msg308137 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-12 15:21
See also https://github.com/python/buildmaster-config/pull/26 : "Add the Android build factory".
msg308143 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-12 15:44
See also the thread on python-dev:
[Python-Dev] Support of the Android platform
https://mail.python.org/pipermail/python-dev/2017-December/151171.html
History
Date User Action Args
2022-04-11 14:58:46adminsetgithub: 74571
2019-12-10 11:59:46xdegayesetstatus: open -> closed
resolution: works for me
stage: patch review -> resolved
2017-12-12 15:44:46vstinnersetmessages: + msg308143
2017-12-12 15:21:53vstinnersetmessages: + msg308137
2017-05-30 18:46:10xdegayesetmessages: + msg294777
2017-05-30 18:29:42xdegayesetmessages: + msg294776
2017-05-27 19:57:12xdegayesetmessages: + msg294610
2017-05-19 13:21:01xdegayesetmessages: + msg293956
2017-05-18 21:58:54zach.waresetnosy: + zach.ware
2017-05-17 14:23:57xdegayesetfiles: + android-api-24.patch
2017-05-17 14:23:36xdegayesetfiles: + issue29619.patch
keywords: + patch
messages: + msg293856
2017-05-17 14:17:08xdegayesetpull_requests: + pull_request1720
2017-05-17 13:57:53xdegayecreate