Here's the error:

test_distutils
test test_distutils failed -- Traceback (most recent call last):
File
"/private/tmp/tmp8UfLPT/python27/Lib/distutils/tests/test_sdist.py",
line 342, in test_make_distribution_owner_group 
    self.assertEquals(member.gid, os.getgid())
AssertionError: 0 != 20

It has been a problem for over a week, at least.

Also seeing it on 10.5 ...

This test makes sure that a tarball created with sidist has the same
owner/group than your current user in each one of its member. Maybe
somehow, the previous tarball creation breaks this test.

Can you try this on your side:

Comment everything in "test_make_distribution_owner_group" between these
lines (318)

   # now building a sdist

   ...

   # building a sdist again

And see if it works.  Thx!

I don't think its OSX specific.  My FreeBSD slaves (both 6.4 and 7.2)
have been getting the same error recently (sounds like probably around
the same time frame).  The error always seems to be that member.gid (0)
is not matching os.getgid (1001).  The os.getgid call is correct in
terms of that being the group under which the build slave is running.

In digging a little deeper I think the the generated distribution is
behaving correctly, just not how the test is expecting.  On my FreeBSD
systems (and my personal OSX 10.4 system for that matter), if I make a
directory in /tmp it gets my UID, but a GID of 0 (wheel).  Whether
that's because my id is also in the wheel group or because of the sticky
bit on /tmp I'm not totally sure.  Maybe it needs both (/tmp is sticky
on my Ubuntu system but my id is not in the root group, and this doesn't
happen).

So the files being created for the second half of this test are placed
in a temporary folder with a group of wheel(0), and the files appear to
get the same ownership.  This doesn't happen elsewhere, but appears to
be special behavior beneath tmp.

Perhaps if the test did a comparison to the UID/GID of tmp_dir rather
than os.getuid()/os.getgid() it would better reflect that the tarfile
was being built with the same ownership as found in the filesystem. 
Whether that's the correct ownership is a separate question, but as long
as distutils is accurately reflecting that it shouldn't be a failure on
a distutils test.

-- David

Thanks for the digging David, I'll check for /tmp rights in that case.
What's important in distutils, is to make sure it was able to create
tarballs with various uid/gid using the new tarfile feature.

David, could it be that the directories where a change of group occurs
also have the s-bit set? The sticky bit should have no such effect.

I can generate the error on my iMac but not my laptop, which are
equivalent versions/upgrades of Mac OS X AFAIK.  The /tmp directory
in both has drwxrwxrwt, and the subdirectories within which I'm doing
the builds are drwxr-xr-x, so it doesn't seem to be only dir permissions...

Both accounts are user accounts with +wheel permissions.

Tarek, after commenting out those lines (starting at line 309 of my
test_sdist.py), I still get the same error on my iMac.

From Tarek:

> Thanks for the digging David, I'll check for /tmp rights in that case.
> What's important in distutils, is to make sure it was able to create
> tarballs with various uid/gid using the new tarfile feature.

Right, and that part of the test (explicitly making uid/gid=0/0) is
passing.  It's just the default case that is failing, and only because
the test's assumption doesn't match what is actually happening in the
filesystem in terms of default group ownership.

From Martin:

> David, could it be that the directories where a change of group occurs
> also have the s-bit set? The sticky bit should have no such effect.

Not sure - I only noticed the "t" suffix on the directory "ls -ald"
listing, which the ls man page called sticky, but didn't think about it
much more than that.

-- David

This does seem to be a long standing Unix*-flavor behavior difference.  

The open(2) man page on OS X (and presumably FreeBSD) reads:

   When a new file is created, it is given the group of the directory
   which contains it.

The Linux open(2) page has:

       O_CREAT
          If the file does not exist it will be created.  The owner  
(user  ID)  of
          the  file is set to the effective user ID of the process.  The 
group own‐
          ership (group ID) is set either to the effective group ID of 
the  process
          or to the group ID of the parent directory (depending on file 
system type
          and mount options, and the mode of the parent directory,  see  
the  mount
          options bsdgroups and sysvgroups described in mount(8)).

The Open Group Base Specifications (Issue 6, 2004) for open(1) has this 
rationale:

"The POSIX.1-1990 standard required that the group ID of a newly created 
file be set to the group ID of its parent directory or to the effective 
group ID of the creating process. FIPS 151-2 required that 
implementations provide a way to have the group ID be set to the group 
ID of the containing directory, but did not prohibit implementations 
also supporting a way to set the group ID to the effective group ID of 
the creating process. Conforming applications should not assume which 
group ID will be used. If it matters, an application can use chown() to 
set the group ID after the file is created, or determine under what 
conditions the implementation will set the desired group ID."

So the portable solution is to change the test to not assume which group 
ID is used.

> I can generate the error on my iMac but not my laptop, which are
> equivalent versions/upgrades of Mac OS X AFAIK.  The /tmp directory
> in both has drwxrwxrwt, and the subdirectories within which I'm doing
> the builds are drwxr-xr-x, so it doesn't seem to be only dir 
> permissions...

No, it seems to be some combination of ... something.  My Ubuntu boxes
all have "t" on the /tmp directory as well, but don't seem to exhibit
this behavior (whether or not I add myself to the "root" group that
owns /tmp).

But on the FreeBSD and OSX systems I can show this just though "mkdir"
from the command line.

At some level, I'm not sure it really matters as long as the test uses
whatever actual filesystem ownership is in place checking for default
values.

-- David

BTW, it's trivial to demonstrate the difference.

On OS X:

$ cd $HOME
$ mkdir test
$ touch test/test
$ chown :musicg test
$ touch test/test2
$ ls -l test
total 0
-rw-r-----  1 nad  staff   0 Nov 29 14:01 test
-rw-r-----  1 nad  musicg  0 Nov 29 14:01 test2

On my linux system, the same test results in:

$ ls -l test
total 0
-rw-r----- 1 nad nad 0 2009-11-29 14:02 test
-rw-r----- 1 nad nad 0 2009-11-29 14:02 test2

> At some level, I'm not sure it really matters as long as the test uses
> whatever actual filesystem ownership is in place checking for default
> values.

I think that's difficult to implement, as the files are not there
anymore when the check is made (only the tarfile); somebody correct
me if I'm wrong. So I agree that the test for group ownership should
be dropped, in the failing case (i.e. where nothing was specified for
the tar command).

> I think that's difficult to implement, as the files are not there
> anymore when the check is made (only the tarfile); somebody correct
> me if I'm wrong.

The files are created during setup, and I think exist through the
duration of the test, only being removed during tearDown (via
TempDirManager mixin) - otherwise this test couldn't be creating two
separate archives from the same sources.

Worst case though, just have setup grab the test file ownership
information and hang onto it in general.  But I think this specific test
could also check filesystem ownership at any point in the test.

I've dropped the gid control and just check the uid.

From Distutils PoV, as long as this uid test pass, I can defer the
complete uid/gid test to the dedicated tarfile tests.

What is important here is being able to check that I can force a
different uid/gid for sdist archives (the first part of my test) and
this works well.

Thanks everyone for the help !

done in r76588, r76590

Fix verified, thanks!

In issue #19544, #6516 was ported to Python 3.4. After doing so, this issue re-emerges.

New changeset f4b364617abc by Jason R. Coombs in branch 'default':
Issue #7408: Forward port limited test from Python 2.7, fixing failing buildbot tests on BSD-based platforms.
http://hg.python.org/cpython/rev/f4b364617abc

I'm declaring this fixed and watching the buildbots for confirmation.