classification
Title: test_getgroups of test_posix can fail on OS X 10.8 if more than 16 groups
Type: Stage: resolved
Components: Macintosh Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: jcea, mlen, ned.deily, python-dev, ronaldoussoren, tebeka
Priority: normal Keywords: patch

Created on 2013-03-27 09:55 by ned.deily, last changed 2014-02-11 22:04 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
getgroups_osx.patch mlen, 2013-07-06 13:09 review
Messages (9)
msg185319 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-03-27 09:55
Due to a change in behavior for OS X 10.8 (seen with 10.8.3), the code added to posixmodule.c in Issue7900 to handle an unlimited number of groups no longer works.  The code depends on the documented behavior of getgroups(2) failing with EINVAL when the grouplist array in the call is too small to hold all groups.  This works correctly for 10.6 and 10.7.  Currently in 10.8, such a call succeeds and truncates to the first grouplist-size groups.  The getgroups function could probably be modified to always call getgroups(0) first to get the real length.  But it seems to be a clear regression in 10.8 and breaks existing code.  I've opened a bug report with Apple about it.  I'll plan to keep this incident open until I hear something back from them.
msg192399 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-07-06 10:05
Have you heard anything from Apple? If you have a radar number I could try to ping someone at Apple.

I'd be in favor of working around this issue in posixmodule.c if they don't respond or if they won't fix the issue in 10.8 (and I wouldn't be surprised if they won't).
msg192436 - (view) Author: Mateusz Lenik (mlen) * Date: 2013-07-06 13:09
I attached a patch implementing a workaround for getgroups(2) call. When compiled for OS X it first calls getgroups with 0 as the first argument to get number of groups and then in subsequent call the code fetches the gids.

This patch applies cleanly on 2.7 and 3.x branches.
msg192513 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-07-07 06:15
Thanks for the patch, Mateusz. That basic approach should work on all current releases (although it would be nice if Apple also fixed the problem).  I'll test it on the various releases.  If you haven't already, please submit a contributor form as described in the developer's guide (http://docs.python.org/devguide/patch.html#licensing).
msg192525 - (view) Author: Mateusz Lenik (mlen) * Date: 2013-07-07 08:53
I signed it today.
msg194149 - (view) Author: Roundup Robot (python-dev) Date: 2013-08-02 04:38
New changeset 6d3b7e0559a0 by Ned Deily in branch '2.7':
Issue #17557: Fix os.getgroups() to work with the modified behavior of
http://hg.python.org/cpython/rev/6d3b7e0559a0

New changeset 0a4afa8833b5 by Ned Deily in branch '3.3':
Issue #17557: Fix os.getgroups() to work with the modified behavior of
http://hg.python.org/cpython/rev/0a4afa8833b5

New changeset 634a8e8816d4 by Ned Deily in branch 'default':
Issue #17557: merge from 3.3
http://hg.python.org/cpython/rev/634a8e8816d4
msg194150 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-08-02 04:43
Committed patch (with slight modifications) for release in 2.7.6, 3.3.3, and 3.4.0.  Thanks again, Mateusz.
msg210959 - (view) Author: Miki Tebeka (tebeka) * Date: 2014-02-11 16:15
I still see this in 3.4rc1

======================================================================
FAIL: test_getgroups (test.test_posix.PosixTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/private/tmp/Python-3.4.0rc1/Lib/test/test_posix.py", line 780, in test_getgroups
    set(posix.getgroups() + [posix.getegid()]))
AssertionError: Items in the second set but not the first:
33
100
204
398
399
msg211028 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-02-11 22:04
Miki, what is the output of the following on the failing system?

id -G && sw_vers
History
Date User Action Args
2014-02-11 22:04:35ned.deilysetmessages: + msg211028
2014-02-11 16:15:26tebekasetnosy: + tebeka
messages: + msg210959
2013-08-02 04:43:31ned.deilysetstatus: open -> closed
versions: - Python 3.2
messages: + msg194150

resolution: fixed
stage: patch review -> resolved
2013-08-02 04:38:09python-devsetnosy: + python-dev
messages: + msg194149
2013-07-07 08:53:12mlensetmessages: + msg192525
2013-07-07 06:15:24ned.deilysetmessages: + msg192513
stage: patch review
2013-07-06 13:09:25mlensetfiles: + getgroups_osx.patch

nosy: + mlen
messages: + msg192436

keywords: + patch
2013-07-06 10:05:45ronaldoussorensetmessages: + msg192399
2013-03-27 16:55:24jceasetnosy: + jcea
2013-03-27 09:55:41ned.deilycreate