This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_getgrouplist may fail on macOS if user in too many groups
Type: behavior Stage: resolved
Components: Library (Lib), macOS Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Dave Page, Jeffrey.Kintscher, barry, brett.cannon, miss-islington, ned.deily, ronaldoussoren, vstinner, xtreak
Priority: normal Keywords: patch

Created on 2018-10-25 19:52 by barry, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13071 merged Jeffrey.Kintscher, 2019-05-02 23:04
PR 14042 merged miss-islington, 2019-06-13 07:01
PR 14043 merged miss-islington, 2019-06-13 07:01
Messages (34)
msg328474 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-25 19:52
It looks like macOS 10.14 Mojave has changed the return value for getgroups().  On 10.13 it returns the set of GIDs give by `id -G` but afaict on 10.14 it returns only the primary GID.

$ python3 -c "import os; print(os.getgroups())"
[101]
$ id -G 
101 503 701 501 12 62 80 502 33 98 100 204 250 395 398

This breaks test_posix.py.
msg328476 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-25 20:00
Hmm, I'm not seeing that behavior with either a freshly-built top of master 3.8 or with the python.org 3.7.1.

$ ./python  -c "import os;print(os.getgroups())"
[20, 12, 61, 79, 80, 81, 98, 33, 100, 204, 250, 395, 398, 399]
$ id -G
20 12 61 79 80 81 98 33 100 204 250 395 398 399
nad@harj:~/Projects/PyDev/active/dev/3x/source$ ./python -m test -w test_posix
Run tests sequentially
0:00:00 load avg: 1.64 [1/1] test_posix

== Tests result: SUCCESS ==

1 test OK.

Total duration: 13 sec 938 ms
Tests result: SUCCESS
$ ./python
Python 3.8.0a0 (heads/master:9e95eb0d60, Oct 25 2018, 15:55:56)
[Clang 10.0.0 (clang-1000.11.45.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D
msg328477 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-25 20:02
$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.14
BuildVersion:	18A391
msg328480 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-25 20:13
Seems to fail for me with every version of Python 3 on Mojave.  In master, it’s test_getgroups().

> Ned Deily <nad@python.org> added the comment:
> 
> $ sw_vers
> ProductName:	Mac OS X
> ProductVersion:	10.14
> BuildVersion:	18A391

Mine is exactly the same.
msg328482 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-25 20:17
OK. When you asy "every version of Python 3", are those all versions you've built yourself?  If so, what ./configure arguments do you use?
msg328486 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-25 20:27
On Oct 25, 2018, at 13:17, Ned Deily <report@bugs.python.org> wrote:
> 
> OK. When you asy "every version of Python 3", are those all versions you've built yourself?  If so, what ./configure arguments do you use?

Yes, built myself from source (both .tar.gz and git tag).  I don’t use any configure arguments, just

./configure && make && make test
msg328490 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-25 20:52
I've also tried building on a vanilla-ish 10.8 VM, both without and with installing the /usr/include headers, and I still don't see the failure you're seeing.  If you look at Modules/posixmodule.c, you'll see there's a fair amount of Apple-specifc or -caused hackery to deal with past and current getgroups() anomalies.  Perhaps you could step through it on your system and see if it's obvious what's happening.

Also, looking back through previous issues with getgroups, in Issue29562 Ronald noted:

"Note that the result of getgroups(2) is fixed on login, while "id -G" reflects the current state of the user database on macOS. Could this explain this failure? That is, have you tried logging out and in again before running the test suite?"
msg328497 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-25 21:33
"on a vanilla-ish 10.8 VM" ?? I meant 10.14, of course.
msg328526 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2018-10-26 08:30
FWIW I don't see the problem either, VM running 10.14.1 (beta) with Python 3.7.0.
msg328579 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-26 17:33
We're wondering if it could be a weird interaction with Active Directory.  This is my work laptop, so it's all integrated with LDAP and whatnot.  I don't have Mojave on my personal laptop yet (maybe this weekend).  I'm guessing that whatever corporate integration is going on is messing with getgroups(2).  Brett said he vaguely remembers something similar, but I don't remember seeing this problem on High Sierra on the same laptop.  I'm beta testing Mojave internally, so maybe this is just a weirdism I should report to our IT.
msg328581 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-26 18:07
It very well could have something to do with Active Directory support.  Here's a little getgroups test I used a while back for a previous getgroups issue.  You should see something similar in the output depending on the number of groups in your id.  If it fails, we can eliminate Python from the equation.


#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>

int main(){
    gid_t grouplist[32];
    int n;
    int gidsetsize;

    for(gidsetsize = 0; gidsetsize < 22; ++gidsetsize)
    {
        n = getgroups(gidsetsize, grouplist);
        printf("calling grouplist with gidsetsize = %i, returns %i\n", gidsetsize, n);
    }
    exit(0);
}


calling grouplist with gidsetsize = 0, returns 14
calling grouplist with gidsetsize = 1, returns -1
calling grouplist with gidsetsize = 2, returns -1
calling grouplist with gidsetsize = 3, returns -1
calling grouplist with gidsetsize = 4, returns -1
calling grouplist with gidsetsize = 5, returns -1
calling grouplist with gidsetsize = 6, returns -1
calling grouplist with gidsetsize = 7, returns -1
calling grouplist with gidsetsize = 8, returns -1
calling grouplist with gidsetsize = 9, returns -1
calling grouplist with gidsetsize = 10, returns -1
calling grouplist with gidsetsize = 11, returns -1
calling grouplist with gidsetsize = 12, returns -1
calling grouplist with gidsetsize = 13, returns -1
calling grouplist with gidsetsize = 14, returns 14
calling grouplist with gidsetsize = 15, returns 14
calling grouplist with gidsetsize = 16, returns 14
calling grouplist with gidsetsize = 17, returns 14
calling grouplist with gidsetsize = 18, returns 14
calling grouplist with gidsetsize = 19, returns 14
calling grouplist with gidsetsize = 20, returns 14
calling grouplist with gidsetsize = 21, returns 14
msg328582 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-26 18:11
Also, I'm assuming you've tried rebooting your system?  :)
msg328603 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-26 20:35
Yes, I've rebooted :)   I've modified your C program a little and here's the code and output.

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>

int main()
{
    gid_t grouplist[32];
    int n;
    int gidsetsize;

    for (gidsetsize = 0; gidsetsize < 22; ++gidsetsize) {
        n = getgroups(gidsetsize, grouplist);
        printf("calling grouplist with gidsetsize = %i, returns %i: ", gidsetsize, n);
        for (int j = 0; j < n; j++) {
            printf("%i ", grouplist[j]);
        }
        printf("\n");
    }
    exit(0);
}

calling grouplist with gidsetsize = 0, returns 15: -483891656 32766 -483891672 32766 -483891728 32766 427353334 1 -483891696 32766 0 0 0 0 -483891672 
calling grouplist with gidsetsize = 1, returns -1: 
calling grouplist with gidsetsize = 2, returns -1: 
calling grouplist with gidsetsize = 3, returns -1: 
calling grouplist with gidsetsize = 4, returns -1: 
calling grouplist with gidsetsize = 5, returns -1: 
calling grouplist with gidsetsize = 6, returns -1: 
calling grouplist with gidsetsize = 7, returns -1: 
calling grouplist with gidsetsize = 8, returns -1: 
calling grouplist with gidsetsize = 9, returns -1: 
calling grouplist with gidsetsize = 10, returns -1: 
calling grouplist with gidsetsize = 11, returns -1: 
calling grouplist with gidsetsize = 12, returns -1: 
calling grouplist with gidsetsize = 13, returns -1: 
calling grouplist with gidsetsize = 14, returns -1: 
calling grouplist with gidsetsize = 15, returns 15: 101 503 701 501 12 62 80 502 33 98 100 204 250 395 398 
calling grouplist with gidsetsize = 16, returns 15: 101 503 701 501 12 62 80 502 33 98 100 204 250 395 398 
calling grouplist with gidsetsize = 17, returns 15: 101 503 701 501 12 62 80 502 33 98 100 204 250 395 398 
calling grouplist with gidsetsize = 18, returns 15: 101 503 701 501 12 62 80 502 33 98 100 204 250 395 398 
calling grouplist with gidsetsize = 19, returns 15: 101 503 701 501 12 62 80 502 33 98 100 204 250 395 398 
calling grouplist with gidsetsize = 20, returns 15: 101 503 701 501 12 62 80 502 33 98 100 204 250 395 398 
calling grouplist with gidsetsize = 21, returns 15: 101 503 701 501 12 62 80 502 33 98 100 204 250 395 398 

% id -G
101 503 701 501 12 62 80 502 33 98 100 204 250 395 398

I've also made a small change to test_posix.py:

diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py
index 86c04b9f32..5074b45fc0 100644
--- a/Lib/test/test_posix.py
+++ b/Lib/test/test_posix.py
@@ -1047,8 +1047,9 @@ class PosixTester(unittest.TestCase):
         # groups, ignoring order, duplicates, and the effective gid.
         # #10822/#26944 - It is implementation defined whether
         # posix.getgroups() includes the effective gid.
-        symdiff = idg_groups.symmetric_difference(posix.getgroups())
-        self.assertTrue(not symdiff or symdiff == {posix.getegid()})
+        groups = posix.getgroups()
+        symdiff = idg_groups.symmetric_difference(groups)
+        self.assertTrue(not symdiff or symdiff == {posix.getegid()}, (idg_groups, groups, symdiff))
 
     # tests for the posix *at functions follow
 
But when I run ./python.exe -m test test.test_posix here's what I get:

Run tests sequentially
0:00:00 load avg: 1.62 [1/1] test.test_posix
test test.test_posix failed -- Traceback (most recent call last):
  File "/Users/bwarsaw/projects/python/cpython/Lib/test/test_posix.py", line 1052, in test_getgroups
    self.assertTrue(not symdiff or symdiff == {posix.getegid()}, (idg_groups, groups, symdiff))
AssertionError: False is not true : ({33, 98, 100, 101, 395, 12, 204, 398, 80, 501, 502, 503, 250, 701, 62}, [101], {33, 98, 100, 395, 12, 204, 398, 80, 501, 502, 503, 250, 701, 62})

So it seems like getgroups(2) is doing the right thing, but not Python.  Weird.
msg328607 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-26 21:44
Yeah. Can you check the header file paths that are being used in your compiler invocation, e.g something like:  cc -v -c ...
And, while you are at it, the library search path of the test executable:  otool -L ./...
And anything suspicious in /usr/local/include or /usr/local/lib ?
msg328610 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-26 21:47
Er, and perhaps the compile and link calls from when Python builds posixmodule.c ?
msg328616 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-26 23:43
% gcc -v gg.c
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin18.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name gg.c -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 409.12 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0 -fdebug-compilation-dir /Users/bwarsaw/projects/python -ferror-limit 19 -fmessage-length 131 -stack-protector 1 -fblocks -fencode-extended-block-signature -fobjc-runtime=macosx-10.14.0 -fmax-type-align=16 -fdiagnostics-show-option -o /var/folders/06/yd7czfp11bx1vx4p5dl10v4w000slb/T/gg-a38469.o -x c gg.c
clang -cc1 version 10.0.0 (clang-1000.11.45.2) default target x86_64-apple-darwin18.0.0
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.14.0 -o a.out /var/folders/06/yd7czfp11bx1vx4p5dl10v4w000slb/T/gg-a38469.o -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/lib/darwin/libclang_rt.osx.a

% otool -L a.out 
a.out:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)

So I guess it's interesting that it's searching /usr/local/include, which has some Homebrew headers in it:

% ls /usr/local/include
ImageMagick-7@		gdcache.h@		libltdl@		pg_config_manual.h@	sqlda-native.h@
adns.h@			gdfontg.h@		libpng16@		pg_config_os.h@		sqlda.h@
aspell.h@		gdfontl.h@		libpq@			pgtypes.h@		tiff.h@
assuan.h@		gdfontmb.h@		libpq-events.h@		pgtypes_date.h@		tiffconf.h@
ecpg_config.h@		gdfonts.h@		libpq-fe.h@		pgtypes_error.h@	tiffio.h@
ecpg_informix.h@	gdfontt.h@		libssh2.h@		pgtypes_interval.h@	tiffio.hxx@
ecpgerrno.h@		gdfx.h@			libssh2_publickey.h@	pgtypes_numeric.h@	tiffvers.h@
ecpglib.h@		gdpp.h@			libssh2_sftp.h@		pgtypes_timestamp.h@	unicase.h@
ecpgtype.h@		gmp.h@			libtasn1.h@		png.h@			uniconv.h@
entities.h@		gmpxx.h@		libusb-1.0@		pngconf.h@		unictype.h@
evdns.h@		gnutls@			ltdl.h@			pnglibconf.h@		unigbrk.h@
event.h@		gpg-error.h@		lzma@			popt.h@			unilbrk.h@
event2@			gpgrt.h@		lzma.h@			postgres_ext.h@		uniname.h@
evhttp.h@		graphviz@		mysql@			pspell@			uninorm.h@
evrpc.h@		idn2.h@			nettle@			python3.6m/		unistdio.h@
evutil.h@		informix@		newt.h@			python3.7m/		unistr.h@
fontconfig@		internal@		npth.h@			python3.8m/		unistring@
freetype2@		jconfig.h@		nspr@			readtags.h@		unitypes.h@
gcrypt.h@		jerror.h@		openjpeg-2.3@		server@			uniwbrk.h@
gd.h@			jmorecfg.h@		p11-kit-1@		slang.h@		uniwidth.h@
gd_color_map.h@		jpeglib.h@		pcre2.h@		slcurses.h@		webp@
gd_errors.h@		ksba.h@			pcre2posix.h@		sql3types.h@
gd_io.h@		lcms2.h@		pg_config.h@		sqlca.h@
gdbm.h@			lcms2_plugin.h@		pg_config_ext.h@	sqlda-compat.h@

All those symlinks point into ../Cellar/

It doesn't look much different than what happens on 10.13.  I'll try to see what the paths are when I build Python.
msg328617 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-27 00:04
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -v   -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration   -I. -I./Include     -DPy_BUILD_CORE -c ./Modules/posixmodule.c -o Modules/posixmodule.o
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin18.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name posixmodule.c -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -dwarf-column-info -debug-info-kind=standalone -dwarf-version=4 -debugger-tuning=lldb -target-linker-version 409.12 -v -coverage-notes-file /Users/bwarsaw/projects/python/cpython/Modules/posixmodule.gcno -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0 -D NDEBUG -I . -I ./Include -D Py_BUILD_CORE -O3 -Wno-unused-result -Wsign-compare -Wunreachable-code -Wall -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -std=c99 -fdebug-compilation-dir /Users/bwarsaw/projects/python/cpython -ferror-limit 19 -fmessage-length 131 -fwrapv -stack-protector 1 -fblocks -fencode-extended-block-signature -fobjc-runtime=macosx-10.14.0 -fmax-type-align=16 -fdiagnostics-show-option -vectorize-loops -vectorize-slp -o Modules/posixmodule.o -x c ./Modules/posixmodule.c
clang -cc1 version 10.0.0 (clang-1000.11.45.2) default target x86_64-apple-darwin18.0.0
#include "..." search starts here:
#include <...> search starts here:
 .
 ./Include
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
./Modules/posixmodule.c:1254:9: warning: code will never be executed [-Wunreachable-code]
        PyErr_SetFromErrno(PyExc_OSError);
        ^~~~~~~~~~~~~~~~~~
1 warning generated.
msg328618 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-27 00:36
I really have no idea what's going on.  I've looked at posixmodule.c and tried to reproduce as best I can in a standalone C program.

In both cases getgroups(0, NULL) returns 15.  In the standalone version, when I then call getgroups(15, grouplist), I again get 15 returned and grouplist is properly filled.  But in posixmodule.c case, I get 1 back group that second call.

I honestly can't see any difference or why this could be happening.
msg328621 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-27 00:50
Yep, stepping through Python with lldb, that's what's happening.  I know one of my coworkers has been able to reproduce it.  I'll chime in next once I upgrade my personal machine and can try it there, since I know it isn't on AD.
msg328624 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-27 00:58
It seems like people have fun with getgroups() on macOS:

* bpo-7900
* bpo-16661
* bpo-17557
* bpo-29562

Maybe we should simply skip test_getgroups() and test_getgrouplist() on macOS with a comment listing all these BPO issues.
msg328625 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-27 01:25
Wonderful
msg328626 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-27 02:52
issue33223 also reports failure of test_posix under 10.13.4 and seems to be related.
msg328847 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-10-29 17:48
I've now updated my personal machine to Mojave and cannot reproduce the failure.  I'm going to chalk this one up to some weird corporate active directory or whatnot weirdness.  I'd still love to know why the code in Python works differently that the standalone C version, but I don't plan on spending much time on this issue unless new data comes in.
msg339967 - (view) Author: Dave Page (Dave Page) Date: 2019-04-11 11:36
I'm seeing what appears to my uneducated eyes to be the same failure on Mojave, on a brand new machine which is entirely standalone:

12:16:00 0:00:07 load avg: 4.24 [133/416/1] test_posix failed
12:16:00 test test_posix failed -- Traceback (most recent call last):
12:16:00   File "/Users/jenkins/workspace/python-macos-build/Python-3.7.2/Lib/test/test_posix.py", line 1026, in test_getgrouplist
12:16:00     self.assertIn(group, posix.getgrouplist(user, group))
12:16:00 OSError: [Errno 25] Inappropriate ioctl for device
12:16:00 0:00:07 load avg: 4.24 [134/416/1] test_ast passed

System info:

pgabf-macos2:Python-3.7.2 jenkins$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.3)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

pgabf-macos2:Python-3.7.2 jenkins$ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: macOS 10.14.3 (18D42)
      Kernel Version: Darwin 18.2.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: pgabf-macos2
      User Name: Jenkins (jenkins)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 10 days 1:30

Happy to provide any additional info or try out patches if anyone wants.
msg340961 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-04-27 03:06
The test fails for me on Mohave when I build using clang 10.0.0, but passes when I build using gcc 8.3.0.:

ProductName:	Mac OS X
ProductVersion:	10.14.3
BuildVersion:	18D109

gcc-8 (Homebrew GCC 8.3.0) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
msg341020 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-04-28 04:14
My mistake. It still failed with gcc 8.3.0.
msg341327 - (view) Author: Dave Page (Dave Page) Date: 2019-05-03 08:09
The submitted patch from websurfer5 resolves the issue for me.
msg344380 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-03 04:51
Thanks for the analysis and the PR!  It turns out the problem is not new to 10.14; as you discovered, it's simply a matter of how may groups a particular user is in and, with all the system-generated groups, that number may vary from release to release and from user to user - which explains why the problem is not always reproducible.  I've added a few review comments to the PR.
msg345461 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-13 07:01
New changeset 8725c83ed5ca8959195ad8326db99d564a921749 by Ned Deily (Jeffrey Kintscher) in branch 'master':
bpo-35070: test_getgrouplist may fail on macOS if too many groups (GH-13071)
https://github.com/python/cpython/commit/8725c83ed5ca8959195ad8326db99d564a921749
msg345464 - (view) Author: miss-islington (miss-islington) Date: 2019-06-13 07:18
New changeset b4c8ef7c67712c6639ee896bf7cb8ca1c204946d by Miss Islington (bot) in branch '3.7':
bpo-35070: test_getgrouplist may fail on macOS if too many groups (GH-13071)
https://github.com/python/cpython/commit/b4c8ef7c67712c6639ee896bf7cb8ca1c204946d
msg345466 - (view) Author: miss-islington (miss-islington) Date: 2019-06-13 07:27
New changeset c80183e6ca8c0ce834fc6444a71c7f31a3eb05b7 by Miss Islington (bot) in branch '3.8':
bpo-35070: test_getgrouplist may fail on macOS if too many groups (GH-13071)
https://github.com/python/cpython/commit/c80183e6ca8c0ce834fc6444a71c7f31a3eb05b7
msg345467 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-13 07:30
Merged for release in 3.8.0b2 and 3.7.4.
msg345470 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-13 07:47
This issue has roamed quite a bit so it is a little bit difficult to tell what problem(s) have been seen here.  But, clearly Jeffrey's PR fixes a real and now reproducible problem so I'm declaring victory and have closed this issue now.  Thanks everyonw!  I am going to leave Issue33223 open (thanks xtreak!) for further investigation.
msg345480 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-13 09:15
> clearly Jeffrey's PR fixes a real and now reproducible problem so I'm declaring victory and have closed this issue now.  Thanks everyone!

Well done Jeffrey Kintscher! The test was failing randomly for years, I recall many bugs and many people complaining, but nobody succeeded to come up with a fix.

Note: this off-by-one looks magic to me, but I'm happy that it works :-)

    /*
     * NGROUPS_MAX is defined by POSIX.1 as the maximum
     * number of supplimental groups a users can belong to.
     * We have to increment it by one because
     * getgrouplist() returns both the supplemental groups
     * and the primary group, i.e. all of the groups the
     * user belongs to.
     */
    ngroups = 1 + MAX_GROUPS;
History
Date User Action Args
2022-04-11 14:59:07adminsetgithub: 79251
2019-06-13 09:15:11vstinnersetmessages: + msg345480
2019-06-13 07:47:57ned.deilysetmessages: + msg345470
2019-06-13 07:30:05ned.deilysetstatus: open -> closed
versions: + Python 3.9
messages: + msg345467

resolution: fixed
stage: patch review -> resolved
2019-06-13 07:27:27miss-islingtonsetmessages: + msg345466
2019-06-13 07:18:29miss-islingtonsetnosy: + miss-islington
messages: + msg345464
2019-06-13 07:01:57miss-islingtonsetpull_requests: + pull_request13905
2019-06-13 07:01:40miss-islingtonsetpull_requests: + pull_request13904
2019-06-13 07:01:35ned.deilysetmessages: + msg345461
2019-06-03 04:51:43ned.deilysettitle: test_posix fails on macOS 10.14 Mojave -> test_getgrouplist may fail on macOS if user in too many groups
messages: + msg344380
versions: - Python 3.5, Python 3.6
2019-05-28 05:13:20Jeffrey.Kintschersettype: behavior
2019-05-03 08:09:24Dave Pagesetmessages: + msg341327
2019-05-02 23:04:09Jeffrey.Kintschersetkeywords: + patch
stage: patch review
pull_requests: + pull_request12987
2019-04-28 04:14:28Jeffrey.Kintschersetmessages: + msg341020
2019-04-27 03:06:05Jeffrey.Kintschersetnosy: + Jeffrey.Kintscher
messages: + msg340961
2019-04-11 11:36:33Dave Pagesetnosy: + Dave Page
messages: + msg339967
2018-10-29 17:48:27barrysetmessages: + msg328847
2018-10-27 02:52:48xtreaksetnosy: + xtreak
messages: + msg328626
2018-10-27 01:25:17barrysetmessages: + msg328625
2018-10-27 00:58:29vstinnersetnosy: + vstinner
messages: + msg328624
2018-10-27 00:50:08barrysetmessages: + msg328621
2018-10-27 00:36:11barrysetmessages: + msg328618
2018-10-27 00:04:46barrysetmessages: + msg328617
2018-10-26 23:43:52barrysetmessages: + msg328616
2018-10-26 21:47:12ned.deilysetmessages: + msg328610
2018-10-26 21:44:19ned.deilysetmessages: + msg328607
2018-10-26 20:35:05barrysetmessages: + msg328603
2018-10-26 18:11:32ned.deilysetmessages: + msg328582
2018-10-26 18:07:19ned.deilysetmessages: + msg328581
2018-10-26 17:33:08barrysetnosy: + brett.cannon
messages: + msg328579
2018-10-26 08:30:30ronaldoussorensetmessages: + msg328526
2018-10-25 21:33:17ned.deilysetmessages: + msg328497
2018-10-25 20:52:47ned.deilysetmessages: + msg328490
2018-10-25 20:27:50barrysetmessages: + msg328486
2018-10-25 20:17:34ned.deilysetmessages: + msg328482
2018-10-25 20:13:53barrysetmessages: + msg328480
2018-10-25 20:02:25ned.deilysetmessages: + msg328477
2018-10-25 20:00:24ned.deilysetmessages: + msg328476
2018-10-25 19:52:22barrysettitle: test_posix -> test_posix fails on macOS 10.14 Mojave
2018-10-25 19:52:09barrycreate