Due to a change in distutils released with Python 2.6.3, packages that 
use setuptools (version 0.6c9, as of this writing), or the easy_install 
command, to build C extension modules fail with a cryptic message ending 
with:
... .egg/setuptools/command/build_ext.py", line 85, in get_ext_filename 
ext = self.ext_map[fullname] KeyError: <module_name>

Among the packages known to be affected include lxml, zope-interface, 
jinja2, and, hence, packages dependent on these packages (e.g. sphinx, 
twisted, etc.).

The issue is described in greater detail in the Distribute project 
tracker:
http://bitbucket.org/tarek/distribute/issue/41/keyerror-_speedups

The solution there was to modify Distribute, a fork of setuptools, to 
accommodate the change in distutils.

An issue has been opened on the setuptools tracker for this problem:
http://bugs.python.org/setuptools/issue85

A discussion on the distutils-sig mailing list starts here:
http://mail.python.org/pipermail/distutils-sig/2009-October/013534.html

The distutils change forces affected users to either:
(1) migrate from setuptools to Distribute, which can generally done by
    simply running:
          easy_install distribute
(2) avoid upgrading to 2.6.3 until a fix for setuptools is released
    (or patch setuptools themselves)
(3) patch or downgrade distutils to restore the previous behavior

The primary purpose of this issue is to serve as a placeholder for users 
searching the Python issue tracker for this failure symptom.

> (3) patch or downgrade distutils to restore the previous behavior

This is not really an option since distutils code was modified to fix
some bugs, with no API change and no backward compat break. 

Users affected with this problems have to ask the Setuptools maintainer
to change the monkey-patches done on distutils and release a new
version, or have to use Distribute that fixes the problem and provide a
setuptools-compatible distribution.

As far as Distutils is concerned, its development and bug fixes will not
be frozen to make sure a 1 year old release of a tool that does
monkey-patches still work. It's up to this project to adapt its patches.

Before we start pointing fingers or throwing mud, was this bug fix which 
apparently breaking setuptools introduced knowing that this would break 
setuptools users?

I'm surprised that nobody at all tried to use the release candidate with 
any of the listed packages.

Given the number of packages that use setuptools and have been widely 
distributed, I expect that even if a fixed version of setuptools was 
released tomorrow, the damage would already be done, and most people 
would just have to avoid 2.6.3.

How hard would it be to release a 2.6.4 which differs from 2.6.3 only by 
not breaking existing setuptools distros?

> Before we start pointing fingers or throwing mud, was this bug 
> fix which apparently breaking setuptools introduced knowing that 
> this would break setuptools users?
>
> I'm surprised that nobody at all tried to use the release candidate 
> with any of the listed packages.

The final release of 2.6.3 was done very shortly after the rc, so 
the community didn't have enough time to test their package with 
the rc.

What should have been done to avoid this (what what was suggested) is to
have a special buildbot for this that would run distutils commands over
a list of packages from the community.

I have a buidbot myself that does this, but it was discontinued lately 
because of a lack of CPU/Bandwidth.

> How hard would it be to release a 2.6.4 which differs from 2.6.3 
> only by not breaking existing setuptools distros?

I could work on this by making a special internal case in Distutils to
avoid calling the patched API that leads to the setuptools bug. This is
not hard and can be done quickly if required.

Done in r75256

Thanks for the quick action Tarek!!!

I appreciate the change being made.  The specific code in setuptools is
relying on an erroneous reading of get_ext_filename()'s docstring (which
as of 2.3 at least, implied to me at least that it is always called with
an extension's full dotted name), and so I will fix it in setuptools as
well.

I would like to point out, however, that subclassing is not
"monkeypatching".  The distutils explicitly documents the ability to
extend and subclass commands, and this particular subclassing was based
on code provided by another distutils user, who'd extended build_ext to
support building dynamic libraries.

It is likely that that user, and anyone else who extended the build_ext
command in a similar way (e.g. numpy's and Twisted's distutils
extensions), could be affected by the change.

So, this was neither a matter of monkeypatching, nor special treatment
for setuptools.  Setuptools is far from the first or last distutils
extension ever written, and build_ext tends to get extended more than
most other commands.  It's important to remember subclassing use cases
when modifying distutils commands; subclasses inherently rely on a
stricter interface than a class's public API.

UPDATE for users: With the change to distutils now added in r75256 and a 
2.6.4 release in the works, option (3) in the original message should now 
read:

(3) patch distutils to restore the previous behavior or
    install Python 2.6.4 which is expected to be released within a
    few weeks

> a 2.6.4 release in the works

Is this going to have a rc1 as usual? Have we decided on the dates both 
for rc1 and final?

yes.  rc1 on 6 oct 2009 and final on 18 oct 2009 (tentative)

This change shows a regression (https://bugs.launchpad.net/bugs/449734),
the setup.py file might not be correct, but it did work before this
change (with plain distutils). Quoting from the report:


Set setup.py file looks like this:
setup(name='update-manager',
...
      ext_modules=[Extension('UpdateManager/fdsend',
                             ['UpdateManager/fdsend/fdsend.c'])],
...

WIth 2.6.4~rc1 the logs read:
copying build/lib.linux-x86_64-2.6/fdsend.so ->
/tmp/buildd/update-manager-0.126.1/debian/tmp/usr/lib/python2.6/dist-packages

With 2.6.3:
copying build/lib.linux-x86_64-2.6/UpdateManager/fdsend.so ->
/home/egon/devel/update-manager/checkout/build-area/update-manager-0.126.1/debian/tmp/usr/lib/python2.6/dist-packages/UpdateManager

So it looks like python is eating a prefix.

http://bugs.python.org/issue7064

is why this change was made.

Reading that, it was done because setuptools assumes that the method
is always called with the full dotted name, when it was just being called
with the last part of it.

However, update-manager isn't using a dotted module path anyway, it's
passing a path.

http://docs.python.org/distutils/setupscript.html#extension-names-and-packages

shows that you would be expected to use

      ext_modules=[Extension('UpdateManager.fdsend',
                             ['UpdateManager/fdsend/fdsend.c'])],

(note . not / in the first line)

Tarek,

Can you please look at this for 2.6.4?  We might need another rc and it
would be better to do that sooner rather than later.

Thanks!

Looking at it now

FYI, a fix allowing setuptools to work with 2.6.3 is now checked in, and
will be released soon (preferably in a few days, unless new bugs turn
up).  If you are experiencing issues with this and would like to try the
fixed version(s), please see the announcement here:

http://mail.python.org/pipermail/distutils-sig/2009-October/013822.html

I am opening another issue to keep a record of the particular case
Matthias brought.

Tarek - could you give the new issue link?

The new issue is : #7115

Verified that the original regression reported against 2.6.3 when using 
setuptools 0.6c9 has been fixed and no longer exists in 2.6.4rc2.