Following from issue7457, in which a single feature was identified to have gone missing in 29a3eda89995, this ticket captures the need to bring the Python 3 codebase up to match Python 2.7.

I went through Python 2.7's Misc/NEWS file and collected the entries for Distutils-related issues that were applied.  Perhaps we can check the individual entries on this list, and see which ones are still present in Python 3.x and which ones got reverted.

I wouldn't trust the NEWS items.  I think the only reliable thing to do is diff each file, unfortunately.

Issue #11104 also made some functionality work in both 2.7 and 3.2, though it's not clear that the problem stemmed from the distutils2 revert.

After spending several hours spelunking, we identified what we believe are the tickets that were backed out in the aforementioned reversion.

issue1180
issue6516
issue7457
issue6466
issue6286

Additionally, issue6377 (renaming .compiler to .compiler_obj) was reverted, but it likely should not be re-applied.

Attached is an export of the etherpad (http://beta.etherpad.org/p/python_2.7_distutil_commits) which we used to keep track of the changes and show our work.

We will flag the above tickets and address each individually.

New changeset e19441e540ca by Jason R. Coombs in branch '3.3':
Issue 19544 and Issue #7457: Restore the read_pkg_file method to distutils.dist.DistributionMetadata accidentally removed in the undo of distutils2.
http://hg.python.org/cpython/rev/e19441e540ca

New changeset 28059d8b395b by Jason R. Coombs in branch 'default':
Merge with 3.3 for Issue #19544 and Issue #7457
http://hg.python.org/cpython/rev/28059d8b395b

Patches for the default branch have been added to issue1180 (option to ignore ~/.pydistutils.cfg) and issue6516 (setting the owner/group in Distutils-built tarballs).  Please double-check those patches; I can apply them.

New changeset 5e98c4e9c909 by Jason R. Coombs in branch '3.3':
Issue #19544 and Issue #6286: Restore use of urllib over http allowing use of http_proxy for Distutils upload command, a feature accidentally lost in the rollback of distutils2.
http://hg.python.org/cpython/rev/5e98c4e9c909

New changeset b1244046f37a by Jason R. Coombs in branch 'default':
Merge with 3.3 for Issue #19544 and Issue #6286. Merge is untested. I was unable to test due to bab0cbf86835.
http://hg.python.org/cpython/rev/b1244046f37a

b1244046f37a appears to have broken buildbots.  See, for example:

http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/2984

Thanks Ned. I did see that and have pushed 394ed9deebd4. I believe that corrects the only test failure.

New changeset b9c9c4b2effe by Andrew Kuchling in branch 'default':
Issue #19544 and Issue #6516: Restore support for --user and --group parameters to sdist command as found in Python 2.7 and originally slated for Python 3.2 but accidentally rolled back as part of the distutils2 rollback. Closes Issue #6516.
http://hg.python.org/cpython/rev/b9c9c4b2effe

New changeset b08868fd5994 by Christian Heimes in branch 'default':
Issue #19544 and Issue #6516: quick workaround for failing builds
http://hg.python.org/cpython/rev/b08868fd5994

New changeset 015463176d2e by Victor Stinner in branch 'default':
Issue #19544, #6516: no need to catch AttributeError on import pwd/grp
http://hg.python.org/cpython/rev/015463176d2e

At changeset 015463176d2e2530e4f07cfbe97e41abac540a57, test_make_distribution_owner_group() was failing on some buildbots. test_distutils works fine on my Linux box, I ran the test as my haypo user and as root.

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%203.x/builds/5780/steps/test/logs/stdio

======================================================================
FAIL: test_make_distribution_owner_group (distutils.tests.test_sdist.SDistTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/distutils/tests/test_sdist.py", line 477, in test_make_distribution_owner_group
    self.assertEquals(member.gid, os.getgid())
AssertionError: 0 != 1002

http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/470/steps/test/logs/stdio

======================================================================
FAIL: test_make_distribution_owner_group (distutils.tests.test_sdist.SDistTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/distutils/tests/test_sdist.py", line 477, in test_make_distribution_owner_group
    self.assertEquals(member.gid, os.getgid())
AssertionError: 0 != 20

New changeset c35311fcc967 by Andrew Kuchling in branch 'default':
Issue #19544 and Issue #1180: Restore global option to ignore  ~/.pydistutils.cfg in Distutils, accidentally removed in backout of distutils2 changes.
http://hg.python.org/cpython/rev/c35311fcc967

> At changeset 015463176d2e2530e4f07cfbe97e41abac540a57, test_make_distribution_owner_group() was failing on some buildbots.

The problem is that tempfile.mkdtemp() creates a directory with the
group 0. Files created in this directory also have the group 0.

The test uses os.getpid() to get the group identifier.

The test should use os.stat(self.tmp_dir).st_gid (and st_uid) instead
of os.getgid(), or the tmp_dir directory. Or the group of the
temporary directory should be changed to os.getgid().

I don't know if distutils should create tarball with the group 0 or
the group of the current user.

Thanks Victor.

Yes, it appears that there's yet another unported issue #7408, a follow-up to #6516.

I believe all identified issues have been ported/fixed.

New changeset 83f12a9593db by Zachary Ware in branch 'default':
Issue #19544, #6516: check ZLIB_SUPPORT, not zlib (which might not be bound)
http://hg.python.org/cpython/rev/83f12a9593db

New changeset 2a872126f4a1 by Zachary Ware in branch 'default':
Issue #19544, #6516: check ZLIB_SUPPORT, not zlib (which might not be bound)
http://hg.python.org/cpython/rev/2a872126f4a1