I am on mac OSX high Sierra 10.13.4. I am using the Apple LLVM version 9.1.0 (clang-902.0.37.1) from the XCode toolchain. 

My tests fail on posix and if run the interpreter I get

--- pylog ---
./python.exe
Python 3.8.0a0 (heads/master:da58533ac6, Apr  3 2018, 16:54:53)
[Clang 9.1.0 (clang-902.0.37.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getgrouplist('wizofe', 20)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 0] Error
--- pylog ---

My system is defined as case-sensitive in the setup.

Note: I get the same error with the version of python installed via homebrew:

(Python 3.6.4 (default, Jan  6 2018, 11:51:59)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin

Here is also the output of the id command, if that is helpful:

(box3) λ /Volumes/pythontesting/cpython/ master id
uid=501(wizofe) gid=20(staff) groups=20(staff),1(daemon),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),204(_developer),450(_xcs),701(1),33(_appstore),100(_lpoperator),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh)

I am also attaching a dtrace log of the two liner that causes the bug.

I get the same error on High Sierra, but not on the Mojave beta.  I also regularly get a FileNotFound error instead (which is why I ended up at this issue in the first place).

I'm not sure yet what causes this, a small reproducer in C works correctly and the code in posixmodule.c looks correct. This might be a platform bug (hence my attempt at writing a reproducer in C).

BTW. There is no attached file

I'm slightly closer to understanding the problem. The value of errno is that of the last error *before* getgrouplist(3) was called, that is, getgrouplist return -1 but does not update errno.

And looking at Apple source code this is a bug in CPython after all: getgrouplist(3) return -1 when the buffer is too small, but does not set errno (see <https://opensource.apple.com/source/Libc/Libc-262/gen/getgrouplist.c.auto.html>). The manpage also doesn't mention setting errno.

I'm pretty sure that the use of posix_error() is wrong here. 

We're running into a similar issue as with getgroups: on macOS a user can be a member of more than NGROUPS_MAX groups.  Because of this I'm not yet sure of the correct fix for this issue.

The easy part is that the call to posix_error in the implementation of getgrouplist should be replaced by raising some other error (or possibly just setting errno to EOVERFLOW before calling posix_error()).

The harder part is what to do about the definition of MAX_GROUPS for this function. Either force it to a larger default on macOS, or add some code to increase the buffer size (because the user cannot select the size of buffer to use). The latter is annoyingly hard because the system gives no indication on what the correct buffer size should be.

BTW. The same is also true on other platforms with getgrouplist(), this is not a macOS specific issue other than that NGROUPS_MAX is bogus there.

See also the discussion and fix merged in Issue35070.