classification
Title: android: add platform.android_ver()
Type: enhancement Stage: patch review
Components: Cross-Build, Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Alex.Willmer, Roman.Evstifeev, lemburg, pmpp, vstinner, xdegaye, yan12125
Priority: normal Keywords: patch

Created on 2016-04-26 13:18 by xdegaye, last changed 2018-02-03 00:58 by pmpp.

Files
File name Uploaded Description Edit
platform.patch xdegaye, 2016-04-26 13:18 review
android_ver.patch yan12125, 2016-05-04 19:47 review
android_ver.patch yan12125, 2016-05-04 20:10 Patch version 2 review
android_ver.patch yan12125, 2016-05-11 11:57 Patch version 3 review
Messages (23)
msg264277 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-04-26 13:18
The attached patch misses a test case.

Also how can we be sure that the '/system/build.prop' file may be guaranteed to exist on all android devices ?
It is difficult to get a reliable information on the android infrastructure when the information does not relate to the java libraries.
msg264317 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-04-26 16:49
> Also how can we be sure that the '/system/build.prop' file may be guaranteed to exist on all android devices ?

This path is hard-coded in BioniC [1]. Though I can't find a document/relevant source codes to guarantee 'ro.build.version.release' and 'ro.build.version.sdk' is always in /system/build.prop

And the format of build.prop is not exactly INI. It supports 'import' clauses. See [2] and load_properties() function in [3]

Other options include calling `getprop` via subprocess or using C level function __system_property_get(). The first approach should always work. It's just somewhat tricky on Android 4.1 [4]. For the second option, a bad news is that it's a private API and was just removed from the latest NDK. Chromium has a workaround for that [5] and CPython may use similar approaches.

[1] https://android.googlesource.com/platform/bionic/+/master/libc/include/sys/_system_properties.h
[2] http://forum.xda-developers.com/android/general/explanation-build-prop-values-t3069341
[3] https://android.googlesource.com/platform/system/core/+/master/init/property_service.cpp
[4] https://github.com/rave-engine/python3-android/pull/10#issuecomment-159151445
[5] https://groups.google.com/a/chromium.org/forum/#!topic/chromium-reviews/keQP6L9aVyU
msg264369 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-04-27 09:59
The android/api-level.h header exists at all the android API levels and define __ANDROID_API__ as, for example at level 21:

    #define __ANDROID_API__ 21

So it is possible to get the sdk version from here, and maybe forget about the release version.
msg264373 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-04-27 11:30
Isn't this macro used in compile time? I thought android_ver() aims to check runtime versions.
msg264374 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-04-27 11:58
If the file is guaranteed to exist on most modern Android platforms, this sounds like a good approach.

Perhaps you could add support to fallback to running getprop instead, if the file doesn't exist or cannot be parsed ?!
msg264376 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-04-27 12:07
Some screenshots showing different file contents:

 * http://www1-lw.xda-cdn.com/files/2013/12/tweak-your-samsung-galaxy-s3s-performance-with-these-build-prop-android-hacks.w654.jpg
 * http://media.apcmag.com/wp-content/uploads/sites/20/2014/04/buildprop-text-file.jpg
 * http://i.imgur.com/hZXQaX9.png

I think exposing `codename` and `incremental` may also make sense to get a complete picture.

`ro.product` also has a lot of useful entries which could be used for some of the other platform APIs such as uname().
msg264377 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-04-27 13:15
I have a WIP patch. [1] It's based on Shiz's patch [2].

[1] https://github.com/yan12125/python3-android/blob/cpython-hg/mk/python/android-misc.patch
[2] https://github.com/rave-engine/python3-android/blob/9bb6420317922c07df405315eea040f9301f7eca/mk/python/3.4.3/python-3.4.3-android-misc.patch
msg264852 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-05-04 19:47
Several questions on implementation:
1. Should Android 4.1 supported? If not pass_fds and logics for deriving it can be removed.
2. I can't find ro.build.version.full on any device/emulator I have access to. Maybe it should be removed due to low popularity?

Some related Android source codes, as my implementation basis:
1. Android 4.1's ANDROID_PROPERTY_WORKSPACE: https://android.googlesource.com/platform/bionic/+/android-4.1.1_r1/libc/bionic/system_properties.c#55
2. The Android Framework assumes property values to be valid UTF-8: https://android.googlesource.com/platform/frameworks/base/+/android-5.1.1_r37/core/jni/android_os_SystemProperties.cpp#49. In fact if I feed a malformed /system/build.prop to DalvikVM, it crashes [1]
3. Google's Android Compatibility Test Suite (CTS) [2] assumes `getprop` binary is in $PATH and can be executed: https://android.googlesource.com/platform/cts/+/android-5.1.1_r37/tests/tests/os/src/android/os/cts/BuildTest.java#110

[1] https://github.com/rave-engine/python3-android/pull/10#issuecomment-159390390
[2] https://source.android.com/compatibility/cts/
msg264854 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-05-04 19:59
On 04.05.2016 21:47, Chi Hsuan Yen wrote:
> 1. Should Android 4.1 supported? If not pass_fds and logics for deriving it can be removed.

According to http://www.droid-life.com/tag/distribution/ the Jelly
Bean versions (4.1, 4.2 and 4.3) still have a sizable market
share, so if it's not too much trouble, I think supporting 4.1 would
be a plus.

> 2. I can't find ro.build.version.full on any device/emulator I have access to. Maybe it should be removed due to low popularity?

Agreed.
msg264856 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-05-04 20:10
Version 2 attached. Removed ro.build.version.full.
msg265145 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-05-08 15:11
Android framework provides an SDK_INT field [1], which parses the value of `ro.build.version.sdk` and defaults to 0 if failed [2]. Should android_ver() return an integer for the `sdk` field, too? It simplifies the usage in #26935 and similar ones.

[1] https://android.googlesource.com/platform/frameworks/base/+/e8579b12a3c5be5fef25fc5a1c8c2c9d43e49347/core/java/android/os/Build.java#183
[2] https://android.googlesource.com/platform/frameworks/base/+/e8579b12a3c5be5fef25fc5a1c8c2c9d43e49347/core/jni/android_os_SystemProperties.cpp#66
msg265294 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-05-11 07:50
IMHO returning an integer for the sdk field would be better. The patch could use shutil.which() to avoid subprocess calls when getprop is not available. It would be better to use a 'try/except ValueError' clause instead of isdigit(). The OSError and UnicodeDecodeError exceptions should not be masked, as well as ValueError if sdk cannot be parsed as an int.
msg265307 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-05-11 11:57
Version 3.

Still using a try-catch clause to enclose the subprocess call, as _syscmd_ver does the same thing.
msg267409 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-06-05 11:50
The ro.kernel.qemu property is set to 1 on an Android emulator and 0 otherwise. I think it should be added to the list.  On some threading tests where the switch interval is set to one microsecond, it is necessary to set the switch interval to a higher value when running on the qemu emulator, see the three issues:
    issue #26939: android: test_functools hangs on armv7
    issue #26940: android: test_importlib hangs on armv7
    issue #26941: android: test_threading hangs on armv7
msg267413 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-06-05 12:19
I guess ro.kernel.qemu is not generic enough to be an item in android_ver(). In tests you can simply use _android_getprop()
msg267415 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-06-05 12:30
Agreed, and it is not part of the versioning scheme either.
msg280147 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-11-06 16:25
I see Xavier de Gaye uses sysconfig.get_config_var('ANDROID_API_LEVEL') is various patches. That's OK if the build-time target API level of CPython matches runtime Android API level. Does CPython force API level matching? If so this issue is no longer necessary.
msg280166 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-11-06 19:42
ANDROID_API_LEVEL can only be used in the Python test suite and platform.android_ver() must be used in the standard library. The reason why ANDROID_API_LEVEL can be used in the tests instead of using the runtime API level is that the Python test suite purpose is not to test the compatibilities betweeen all the possible subjacent bionic libc(s) and the tests need only be run on the platform for which they have been built.

platform.android_ver() is a welcome enhancement, and since we are at 3.6 beta now it's too late for 3.6 and it will be implemented in 3.7. Sorry about this delay.
msg307662 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-05 13:19
Chromium still uses the private __system_property_get() function:

https://chromium.googlesource.com/chromium/+/trunk/base/sys_info_android.cc

I would prefer a C function call rather than spawning a subprocess, just to get a value. But the function seems private, and I see discussion about removal of the function...
msg307667 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-12-05 15:46
I cannot find a reference to the "build.prop" file in the Android documentation but google answers at the question ' what is "build.prop"':

    The “build.prop” file is a system file that exists on every Android device. The file contains build information and other system properties which are used throughout the operating system.

and stackoverflow is thriving with questions on how to edit this file.

The file contains part of the information returned by the getprop command and all the information we are needing here.

Here is its content and access rights on an android-24-x86_64 emulator:

generic_x86_64:/data/local/tmp/python $ ls -al /system/build.prop
-rw-r--r-- 1 root root 2083 2017-07-12 22:01 /system/build.prop
generic_x86_64:/data/local/tmp/python $ cat /system/build.prop

# begin build properties
# autogenerated by buildinfo.sh
ro.build.id=NYC
ro.build.display.id=sdk_phone_x86_64-userdebug 7.0 NYC 4174735 test-keys
ro.build.version.incremental=4174735
ro.build.version.sdk=24
ro.build.version.preview_sdk=0
ro.build.version.codename=REL
ro.build.version.all_codenames=REL
ro.build.version.release=7.0
ro.build.version.security_patch=2017-06-05
ro.build.version.base_os=
ro.build.date=Wed Jul 12 19:46:52 UTC 2017
ro.build.date.utc=1499888812
ro.build.type=userdebug
ro.build.user=android-build
ro.build.host=wphl5.hot.corp.google.com
ro.build.tags=test-keys
ro.build.flavor=sdk_phone_x86_64-userdebug
ro.product.model=Android SDK built for x86_64
ro.product.brand=Android
ro.product.name=sdk_phone_x86_64
ro.product.device=generic_x86_64
ro.product.board=
# ro.product.cpu.abi and ro.product.cpu.abi2 are obsolete,
# use ro.product.cpu.abilist instead.
ro.product.cpu.abi=x86_64
ro.product.cpu.abilist=x86_64,x86
ro.product.cpu.abilist32=x86
ro.product.cpu.abilist64=x86_64
ro.product.manufacturer=unknown
ro.product.locale=en-US
ro.wifi.channels=
ro.board.platform=
# ro.build.product is obsolete; use ro.product.device
ro.build.product=generic_x86_64
ro.product.cpu.abilist=x86_64,x86
ro.product.cpu.abilist32=x86
ro.product.cpu.abilist64=x86_64
ro.product.manufacturer=unknown
ro.product.locale=en-US
ro.wifi.channels=
ro.board.platform=
# ro.build.product is obsolete; use ro.product.device
ro.build.product=generic_x86_64
# Do not try to parse description, fingerprint, or thumbprint
ro.build.description=sdk_phone_x86_64-userdebug 7.0 NYC 4174735 test-keys
ro.build.fingerprint=Android/sdk_phone_x86_64/generic_x86_64:7.0/NYC/4174735:userdebug/test-keys
ro.build.characteristics=emulator
# end build properties
#
# from build/target/board/generic_x86_64/system.prop
#
#
# system.prop for generic sdk
#

rild.libpath=/system/lib/libreference-ril.so
rild.libargs=-d /dev/ttyS0

#
# ADDITIONAL_BUILD_PROPERTIES
#
ro.config.notification_sound=OnTheHunt.ogg
ro.config.alarm_alert=Alarm_Classic.ogg
persist.sys.dalvik.vm.lib.2=libart.so
dalvik.vm.isa.x86_64.variant=x86_64
dalvik.vm.isa.x86_64.features=default
dalvik.vm.isa.x86.variant=x86
dalvik.vm.isa.x86.features=default
dalvik.vm.lockprof.threshold=500
xmpp.auto-presence=true
ro.config.nocheckin=yes
net.bt.name=Android
dalvik.vm.stack-trace-file=/data/anr/traces.txt
msg307672 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-05 17:53
"-rw-r--r-- 1 root root 2083 2017-07-12 22:01 /system/build.prop" so
anyone can *read* the file. Maybe we can write a cheap parser for it,
instead of spawning a subprocess or relying on a private function
which may be removed in the near future?
msg307685 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2017-12-05 21:10
Yes, and fall back to spawning getprop if that fails.

In the following link Robin Gawenda is reporting that /system/build.prop is world readable on all the devices he checked: https://stackoverflow.com/questions/9937099/how-to-get-the-build-prop-values
msg308040 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-11 13:00
> Yes, and fall back to spawning getprop if that fails.

I suggest to start simple. Parsing a text file is simple, spawning a subprocess can have annoying side effects :-(
History
Date User Action Args
2018-02-03 00:58:43pmppsetnosy: + pmpp
2017-12-11 13:00:12vstinnersetmessages: + msg308040
2017-12-05 21:10:30xdegayesetmessages: + msg307685
2017-12-05 17:53:16vstinnersetmessages: + msg307672
2017-12-05 16:34:30xdegayelinkissue32210 dependencies
2017-12-05 15:46:51xdegayesetmessages: + msg307667
2017-12-05 13:19:50vstinnersetnosy: + vstinner
messages: + msg307662
2017-02-03 18:47:20xdegayesetassignee: xdegaye ->
2016-11-16 07:37:17xdegayeunlinkissue26935 dependencies
2016-11-06 19:42:12xdegayesetversions: + Python 3.7, - Python 3.6
messages: + msg280166

assignee: xdegaye
components: + Library (Lib)
stage: patch review
2016-11-06 16:25:14yan12125setmessages: + msg280147
2016-10-28 07:15:16xdegayeunlinkissue26939 dependencies
2016-07-09 11:49:58xdegayeunlinkissue27027 dependencies
2016-06-05 12:30:27xdegayesetmessages: + msg267415
2016-06-05 12:19:51yan12125setmessages: + msg267413
2016-06-05 11:56:32xdegayelinkissue26939 dependencies
2016-06-05 11:50:40xdegayesetmessages: + msg267409
2016-05-22 11:40:19xdegayeunlinkissue26944 dependencies
2016-05-21 06:52:21xdegayelinkissue27027 dependencies
2016-05-11 11:57:53yan12125setfiles: + android_ver.patch

messages: + msg265307
2016-05-11 07:50:37xdegayesetmessages: + msg265294
2016-05-08 15:11:36yan12125setmessages: + msg265145
2016-05-08 11:05:21martin.panterlinkissue26935 dependencies
2016-05-04 20:10:43yan12125setfiles: + android_ver.patch

messages: + msg264856
2016-05-04 19:59:03lemburgsetmessages: + msg264854
2016-05-04 19:47:19yan12125setfiles: + android_ver.patch

messages: + msg264852
2016-05-04 08:08:58serhiy.storchakalinkissue26944 dependencies
2016-05-03 07:19:05xdegayesettitle: add platform.android_ver() for android -> android: add platform.android_ver()
2016-04-27 13:15:37yan12125setmessages: + msg264377
2016-04-27 12:07:10lemburgsetmessages: + msg264376
2016-04-27 11:58:30lemburgsetnosy: + lemburg
messages: + msg264374
2016-04-27 11:30:49yan12125setmessages: + msg264373
2016-04-27 09:59:46xdegayesetmessages: + msg264369
2016-04-26 16:49:20yan12125setnosy: + yan12125
messages: + msg264317
2016-04-26 16:27:29Roman.Evstifeevsetnosy: + Roman.Evstifeev
2016-04-26 16:04:41zach.warelinkissue26865 dependencies
2016-04-26 13:18:31xdegayecreate