classification
Title: test_shutil fails when gid name is missing
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Dinesh Wijekoon, Matthias Braun, dino.viehland, ezio.melotti, iritkatriel, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-01-09 21:10 by Dinesh Wijekoon, last changed 2020-11-20 15:13 by iritkatriel. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19032 merged python-dev, 2020-03-16 20:25
Messages (6)
msg257858 - (view) Author: Dinesh Wijekoon (Dinesh Wijekoon) Date: 2016-01-09 21:10
./python.exe -m test -j3 -v test_shutil

The above test fails when id name is missing. The fail message is 
group = grp.getgrgid(gid)[0]
KeyError: 'getgrgid(): gid not found: 203135016'

Following is the results from console "id" command, when the failure happens. 

uid=1336551206(athukora) gid=203135016 groups=203135016,402(com.apple.sharepoint.group.2),12(everyone),62(netaccounts),80(admin),401(com.apple.sharepoint.group.1),33(_appstore),98(_lpadmin),100(_lpoperator),204(_developer),398(com.apple.access_screensharing),399(com.apple.access_ssh)

And again when I change user to sudo the "id" command returns the following and test get passed.
uid=0(root) gid=0(wheel) groups=0(wheel),1(daemon),2(kmem),3(sys),4(tty),5(operator),8(procview),9(procmod),12(everyone),20(staff),29(certusers),61(localaccounts),80(admin),401(com.apple.sharepoint.group.1),402(com.apple.sharepoint.group.2),33(_appstore),98(_lpadmin),100(_lpoperator),204(_developer),398(com.apple.access_screensharing),399(com.apple.access_ssh)

PS:

We tried to debug this a bit and found the bug is possibly in Modules/grpmodule.c, may be at method grp_getgrall_impl. Adding "PyString_AsString(v);" before if command " PyList_Append(d, v) != 0" will fix the issue in test. But we have no idea how its working with this change.
msg257859 - (view) Author: Dinesh Wijekoon (Dinesh Wijekoon) Date: 2016-01-09 21:13
This reported bug occurred in Mac OS.
msg258026 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2016-01-11 21:49
We looked at this at the PyLadies Sprint in Helsinki.
When the group has an id but not a name (see "gid=203135016" in the "id" output -- the name is missing), its entry is missing from the list returned by grp.getgrall().

If this behaviour is expected and desired, then the test should be fixed to handle cases where the name is missing and the KeyError is raised.
Otherwise, the entry could still be included with "None" as name.
msg364356 - (view) Author: Matthias Braun (Matthias Braun) * Date: 2020-03-16 20:44
I have the same issue on my work macbook which has some form of network login configured. The primary group id of my user account there cannot be found in the group database and cannot be resolved to a name.

I'm not sure why that is but I am pretty sure there is no obligation that a gid must be resolvable to a name. I submitted a PR to just skip the last part of the test_shutil/chown if the uid/gid cannot be resolved to a name.
msg364445 - (view) Author: Dino Viehland (dino.viehland) * (Python committer) Date: 2020-03-17 16:51
New changeset 52268941f37e3e27bd01792b081877ec3bc9ce12 by Matthias Braun in branch 'master':
bpo-26067: Do not fail test_shutil / chown when gid/uid cannot be resolved (#19032)
https://github.com/python/cpython/commit/52268941f37e3e27bd01792b081877ec3bc9ce12
msg378373 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-10 09:58
This seems complete, can it be closed?
History
Date User Action Args
2020-11-20 15:13:08iritkatrielsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-10-10 09:58:43iritkatrielsetnosy: + iritkatriel
messages: + msg378373
2020-03-17 16:51:52dino.viehlandsetnosy: + dino.viehland
messages: + msg364445
2020-03-16 20:44:11Matthias Braunsetnosy: + Matthias Braun
messages: + msg364356
2020-03-16 20:25:59python-devsetkeywords: + patch
nosy: + python-dev

pull_requests: + pull_request18381
stage: needs patch -> patch review
2016-01-11 21:49:23ezio.melottisetnosy: + serhiy.storchaka
messages: + msg258026

type: behavior
stage: needs patch
2016-01-09 21:13:46Dinesh Wijekoonsetmessages: + msg257859
2016-01-09 21:10:35Dinesh Wijekooncreate