Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a build infrastructure for Android #74571

Closed
xdegaye mannequin opened this issue May 17, 2017 · 8 comments
Closed

Add a build infrastructure for Android #74571

xdegaye mannequin opened this issue May 17, 2017 · 8 comments
Labels
3.7 (EOL) end of life build The build process and cross-build type-feature A feature request or enhancement

Comments

@xdegaye
Copy link
Mannequin

xdegaye mannequin commented May 17, 2017

BPO 30386
Nosy @ncoghlan, @vstinner, @xdegaye, @vadmium, @zware, @moreati
PRs
  • bpo-30386: Add a build infrastructure for Android. #1629
  • Files
  • issue29619.patch
  • android-api-24.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-12-10.11:59:46.290>
    created_at = <Date 2017-05-17.13:57:53.800>
    labels = ['type-feature', '3.7', 'build']
    title = 'Add a build infrastructure for Android'
    updated_at = <Date 2019-12-10.11:59:46.289>
    user = 'https://github.com/xdegaye'

    bugs.python.org fields:

    activity = <Date 2019-12-10.11:59:46.289>
    actor = 'xdegaye'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-12-10.11:59:46.290>
    closer = 'xdegaye'
    components = ['Cross-Build']
    creation = <Date 2017-05-17.13:57:53.800>
    creator = 'xdegaye'
    dependencies = []
    files = ['46870', '46871']
    hgrepos = []
    issue_num = 30386
    keywords = ['patch']
    message_count = 8.0
    messages = ['293852', '293856', '293956', '294610', '294776', '294777', '308137', '308143']
    nosy_count = 6.0
    nosy_names = ['ncoghlan', 'vstinner', 'xdegaye', 'martin.panter', 'zach.ware', 'Alex.Willmer']
    pr_nums = ['1629']
    priority = 'normal'
    resolution = 'works for me'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue30386'
    versions = ['Python 3.7']

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 17, 2017

    This PR adds a build infrastructure for Android on linux that allows a developer to check that source code changes do not break the existing state of this support by testing the changes on Android with a simple command. For example after a change to the socket module:
    $ /path/to/python_src/Android/makesetup && make python PYTHON_ARGS="-m test test_socket"

    The README in the PR describes how to install the Android NDK and SDK, how to cross-compile Python, how to run python interactively or to run the Python test suite on the emulator, how to debug python on the emulator with gdb.

    This build system has been tested with android-ndk-r13 and android-ndk-r14 which is the latest stable release of the NDK. UnifiedHeaders [1] introduced in android-ndk-r14 is not yet used by this version of the build system.

    [1] https://android.googlesource.com/platform/ndk.git/+/master/docs/UnifiedHeaders.md

    @xdegaye xdegaye mannequin added 3.7 (EOL) end of life build The build process and cross-build type-feature A feature request or enhancement labels May 17, 2017
    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 17, 2017

    The build system for the x86_64, x86 and armv7 architectures has been tested on API 21 and for the x86_64 and arm64 architectures on API 24. Of these combinations of architecture/api, only x86_64 on API 21 builds correctly today, the others were building correctly few weeks ago but recent changes in Python seem to have broken those builds.

    To fix this temporarily:

    • Build the 32 bits architectures (x86 and armv7) with the attached bpo-29619.patch. This is a hack, the problem will be fixed in bpo-29619.
    • Build on API 24 with the attached android-api-24.patch. This is another hack, I will enter a new issue for those problems which are of a kind that is recurrent with the NDK: a function implemented in libc and whose declaration is missing in the NDK headers, a problem that is supposed to be fixed by UnifiedHeaders.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 19, 2017

    Commited 'bpo-30386: Support the new NDK Unified Headers'.

    This does fix the missing declarations in the NDK headers and android-api-24.patch is not needed now.
    It also fixes the missing declaration of sethostname() at API 24 that caused the import of the _socket module to fail.
    More surprisingly it also fixes the compilation failure with 32 bits architectures (the size of off_t is now 8 instead of 4 previously) and bpo-29619.patch is not needed now.
    Only one version of the NDK is supported, currently it is android-ndk-r14 the latest stable release.
    The '-Wno-nullability-completeness' compilation flag is needed to avoid a flow of warnings in the processing of some NDK headers.

    Unified Headers comes with a new annoying problem: langinfo.h exists now but nl_langinfo() is intentionaly not declared except for the __ANDROID_API_FUTURE__ API level
    which is reserved for the development of the NDK itself. The solution chosen here is:

    • Undefine HAVE_LANGINFO_H and CODESET in Include/pyport.h to maintain the status quo in the Python code.
    • The Makefile generated by makesetup runs a script at configure time that checks that nl_langinfo() is not declared and aborts the build otherwise. This allows to detect when langinfo.h starts being not broken anymore when swicthing to a new version of the NDK. In that case the following steps should be taken upon swicthing to this new version:
      + Remove this script from the build system (obviously).
      + Remove the changes in Include/pyport.h.
      + Revert the changes made in bpo-28596.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 27, 2017

    Details of commit "Add Makefile targets to implement the buildbot steps" in 543a287:

    The following buildbot steps, as defined by the UnixBuild class of the master.cfg buildbot configuration file, have now their corresponding commands:

    • Configure: makesetup configureFlags && make configure
    • Compile: make host
    • Test: make buildbottest
    • Clean: make hostclean

    The 'configure' Makefile target builds the native interpreter, the external libraries and runs configure for the target host on its first execution. Afterwards it mainly runs configure. As it is, the native interpreter is not distclean(ed) by the above four steps and therefore changes made in Setup.dist are ignored by the successive rebuilds of the native interpreter.
    The 'buildbottest' Makefile target invokes the 'buildbottest' target of the host Python Makefile and runs using also the variables: TESTPYTHONOPTS, TESTTIMEOUT, TESTOPTS.

    The following changes have also been made:

    • The new Makefile target 'kill_emulator' allows to shutdown the emulator.
    • All the shell and python scripts are now copied to the emulator at <sys.exec_prefix>/bin (i.e. /data/local/tmp/python/bin).
    • The python_shell.sh script run by the user in interactive sessions ('install', 'emulator', 'adb_shell') and the shell scripts run by the 'python' and 'buildbottest' targets now change the current directory to <sys.exec_prefix> (i.e. /data/local/tmp/python) instead of its parent previously. As a result, the temporary files and directories left over by 'buildbottest' are removed on the next run with the removal of the content of the previous <sys.exec_prefix> directory before the new installation.
    • Fix file names whose size exceed MAX_FNAME_SIZE.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 30, 2017

    Details of commit "Update the native interpreter build on Setup.dist and version changes." in 408c05d:

    Native interpreter build:

    • Copy Setup.dist to Setup when Setup.dist has been changed.
    • Run 'make distclean' upon a Python version change.
      Also remove the Misc/NEWS entry until the final commit to avoid merge conflicts.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 30, 2017

    I don't intend to change the current code in PR 1629 until there is a review or comments asking for changes. Ned Deily has self-requested a review, reviews from other people are also welcome.

    @vstinner
    Copy link
    Member

    See also python/buildmaster-config#26 : "Add the Android build factory".

    @vstinner
    Copy link
    Member

    See also the thread on python-dev:
    [Python-Dev] Support of the Android platform
    https://mail.python.org/pipermail/python-dev/2017-December/151171.html

    @xdegaye xdegaye mannequin closed this as completed Dec 10, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life build The build process and cross-build type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant