classification
Title: test_posix: Android 'id -G' is entirely wrong or missing the effective gid
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: xdegaye Nosy List: Alex.Willmer, larry, python-dev, r.david.murray, serhiy.storchaka, vstinner, xdegaye, yan12125
Priority: normal Keywords: patch

Created on 2016-05-04 08:01 by xdegaye, last changed 2017-03-31 16:36 by dstufft. This issue is now closed.

Files
File name Uploaded Description Edit
test_getgroups.patch xdegaye, 2016-05-22 11:40 review
test_getgroups_2.patch xdegaye, 2016-07-21 15:54 review
test_getgroups_3.patch r.david.murray, 2016-08-19 00:09 review
test_getgroups_4.patch xdegaye, 2016-10-12 15:10 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017-03-31 16:36
Messages (15)
msg264789 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-05-04 08:01
test_posix fails on an android emulator running an x86 system image at API level 21.

On android we have instead of a list of group IDs:
root@generic_x86:/data/local/tmp # id -G
uid=0(root) gid=0(root) groups=1003(graphics),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats) context=u:r:su:s0


======================================================================
ERROR: test_getgroups (test.test_posix.PosixTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_posix.py", line 815, in test_getgroups
    set([int(x) for x in groups.split()]),
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_posix.py", line 815, in <listcomp>
    set([int(x) for x in groups.split()]),
ValueError: invalid literal for int() with base 10: 'uid=0(root)'

----------------------------------------------------------------------
Ran 83 tests in 0.114s

FAILED (errors=1, skipped=11)
test test_posix failed
1 test failed:
    test_posix
Total duration: 0:00:01
msg264793 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-04 08:08
See more information in comments in issue26932. Since the id tool looks fixed in Android 6.x we should skip the test on Android <6.x.
msg265251 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-05-10 15:09
'id -G' produces the expected result on an android-23-x86 emulator running Android 6.0 (API 23).

There is a new error now instead of the one reported at issue #26932:

======================================================================
FAIL: test_getgroups (test.test_posix.PosixTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_posix.py", line 816, in test_getgroups
    set(posix.getgroups() + [posix.getegid()]))
AssertionError: Items in the second set but not the first:
0


The problem is that posix.getegid() returns 0, and 0 is not in the list printed by 'id -G'.
msg266070 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-05-22 11:40
test_getgroups is ok on an android emulator running API 23 with the attached patch, and the test is skipped when running API 21.
msg270936 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-07-21 15:54
New patch with cosmetic changes.
msg270937 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-07-21 15:56
Is there a plan to make Android a supported platform in 3.6?
msg270943 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-07-21 17:05
> Is there a plan to make Android a supported platform in 3.6?

Answer in the Android meta-issue at msg 270942.
msg271257 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-07-25 12:22
> ValueError: invalid literal for int() with base 10: 'uid=0(root)'

Hum, does the id program supports -G on Android? The output looks like the regular id output (without -G). Example on my Linux (Fedora 24):

$ id
uid=1000(haypo) gid=1000(haypo) groupes=1000(haypo),10(wheel) contexte=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

$ id -G
1000 10
msg271273 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-07-25 14:01
> Hum, does the id program supports -G on Android?

It does on Android 6.0 but prints the same result as 'id' (without -G) on previous Android versions.

Here is the output of the commands involved in the test for the root user on my archlinux box, the Android 5.0 emulator and the Android 6.0 emulator:

archlinux:
  [root@bilboquet default]# python
  ...
  >>> from posix import getegid, getgroups
  >>> getgroups()
  [0, 1, 2, 3, 4, 6, 10, 19]
  >>> getegid()
  0
  [root@bilboquet default]# id -G
  0 1 2 3 4 6 10 19

Android 5.0 (API level 21)
  root@generic_x86:/data/data/org.bitbucket.pyona # python
  ...
  >>> from posix import getegid, getgroups
  >>> getgroups()
  [1003, 1004, 1007, 1011, 1015, 1028, 3001, 3002, 3003, 3006]
  >>> getegid()
  0
  root@generic_x86:/data/data/org.bitbucket.pyona # id -G
  uid=0(root) gid=0(root) groups=1003(graphics),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats)

Android 6.0 (API level 23)
  root@generic_x86:/data/data/org.bitbucket.pyona # python
  ...
  >>> from posix import getegid, getgroups
  >>> getgroups()
  [1004, 1007, 1011, 1015, 1028, 3001, 3002, 3003, 3006]
  >>> getegid()
  0
  root@generic_x86:/data/data/org.bitbucket.pyona # id -G
  1004 1007 1011 1015 1028 3001 3002 3003 3006
msg271280 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-07-25 15:12
Sorry but I don't see the point of trying to "fix" the unit test on
Android if "id -G" doesn't work as expected.

In fact, I don't really understand the point of such functional test.
Are we still testing Python? Or are we testing the OS itself?

I suggest to remove the whole unit test, or skip the whole unit test
on Android (@skipIf(is_android(), "-G option of id -G is ignored"),
something like that).
msg271285 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-07-25 15:25
Ooops, I missed the part where you say that it works on API level >= 23. So I suggest to only skip the test on Android API level < 23.
msg271291 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-07-25 16:00
The test is already skipped by the patch, for Android API level < 23, with the statement:
    raise unittest.SkipTest("need working 'id -G'")

Do you mean that the test should be skipped instead more explicitly with something like:
    skiIf(support.android_api_level < 23, "-G option of id -G is ignored by Android")
msg273068 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-08-19 00:09
Haypo: we are testing that our function wrapper (getgroups) returns something that we can parse as matching the results from the 'id' command.  This gives us a much more meaningful test than just testing that getgroups returns *something*.  That is, we are testing that it returns integers, and that they "look right".

In that light, I propose an alternate patch.
msg278531 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-10-12 15:10
Thanks for the patch David, see my comment in Rietveld.
New patch based on David patch, just a bit simpler.
msg278969 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-19 09:09
New changeset fb3e65aff225 by Xavier de Gaye in branch '3.6':
Issue #26944: Fix test_posix for Android where 'id -G' is entirely wrong
https://hg.python.org/cpython/rev/fb3e65aff225

New changeset 465c09937e85 by Xavier de Gaye in branch 'default':
Issue #26944: Merge with 3.6.
https://hg.python.org/cpython/rev/465c09937e85
History
Date User Action Args
2017-03-31 16:36:11dstufftsetpull_requests: + pull_request866
2016-10-19 09:11:29xdegayesetstatus: open -> closed
resolution: fixed
stage: commit review -> resolved
2016-10-19 09:09:05python-devsetnosy: + python-dev
messages: + msg278969
2016-10-19 08:10:04xdegayesettitle: android: test_posix fails -> test_posix: Android 'id -G' is entirely wrong or missing the effective gid
2016-10-19 08:06:30xdegayesetdependencies: - add the 'is_android' attribute to test.support
2016-10-12 15:10:59xdegayesetfiles: + test_getgroups_4.patch

stage: patch review -> commit review
messages: + msg278531
versions: + Python 3.7
2016-08-19 00:12:06r.david.murraysetstage: commit review -> patch review
2016-08-19 00:09:51r.david.murraysetfiles: + test_getgroups_3.patch
nosy: + r.david.murray
messages: + msg273068

2016-07-25 16:00:52xdegayesetmessages: + msg271291
2016-07-25 15:25:31vstinnersetmessages: + msg271285
2016-07-25 15:12:47vstinnersetmessages: + msg271280
2016-07-25 14:01:10xdegayesetmessages: + msg271273
2016-07-25 12:22:38vstinnersetmessages: + msg271257
2016-07-24 16:46:42xdegayesetnosy: + vstinner

stage: patch review -> commit review
2016-07-21 17:05:38xdegayesetmessages: + msg270943
2016-07-21 15:56:35larrysetmessages: + msg270937
2016-07-21 15:54:56xdegayesetfiles: + test_getgroups_2.patch
assignee: xdegaye
messages: + msg270936

stage: patch review
2016-05-22 11:40:19xdegayesetfiles: + test_getgroups.patch
keywords: + patch
dependencies: + add the 'is_android' attribute to test.support, - android: add platform.android_ver()
messages: + msg266070
2016-05-21 07:06:39xdegayelinkissue26865 dependencies
2016-05-10 15:09:27xdegayesetnosy: + yan12125
messages: + msg265251
2016-05-04 08:08:58serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg264793

dependencies: + android: add platform.android_ver()
components: + Tests, - Library (Lib), Cross-Build
type: behavior
2016-05-04 08:01:13xdegayecreate