Title: try to build a C module, but don't worry if it doesn't work
Type: enhancement Stage:
Components: Distutils Versions: Python 3.1, Python 2.7
Status: closed Resolution: duplicate
Dependencies: Superseder: Optional extensions in
View: 5583
Assigned To: tarek Nosy List: exarkun, tarek, zooko
Priority: normal Keywords:

Created on 2008-12-20 14:09 by zooko, last changed 2009-03-31 21:59 by tarek. This issue is now closed.

File name Uploaded Description Edit
build-with-no-Python.h.txt zooko, 2009-02-01 02:20
build-with-no-gcc.txt zooko, 2009-02-01 02:21
Twisted-build-with-no-gcc.txt zooko, 2009-02-01 02:27
Twisted-build-no-Python.h.txt zooko, 2009-02-01 02:59
Messages (8)
msg78106 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2008-12-20 14:09
I was trying to install Twisted on my son's OLPC.  Twisted comes with a
handful of C extensions which are all optional and most of which are not
expected to compile on this platform anyway (they are platform-specific
for Mac or Windows).  If my son's OLPC had a C compiler installed, then
it would attempt to build those modules, fail, and proceed to install
Twisted.  However, since it doesn't have a C compiler, then instead it
stops with an error.

It would be nice if Twisted could tell distutils to try to build a
module, and report whether the build succeeded, but don't stop the
entire setup().
msg79047 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2009-01-04 11:43
I can see two cases here:

1/ a compiler is not found, and build_ext can't proceed
2/ a compiler is found but it fails

For 2/, I don't think it's wise to fail silently if a compiler is found
and the build fails, because that can hide an unexpected problem.

For 1/ I would go for a binary distribution, or for a source
distribution that is specific to Linux in your case, if the extensions
are specific to other platforms. The latter could be done in the same if needed by declaring the extensions only on matching platforms.

But I can see the benefit of a source distribution with optional
extensions in general. In that case, I think this option would have to
be declared optionally for each extension to be able to fail properly if
one extension is mandatory.
msg80681 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2009-01-28 00:03

Yes, failure to build an extension module for any reason should
definitely cause "build" to fail by default.  However, many Python
packages seem to come with optional extension modules, typically for a
performance speed-up.  simplejson, zope.interface, and twisted all do
this (these are three packages that allmydata-tahoe depends on).  My own
pyutil package used to do this, but I eventually just removed the
optional C extension module in order to avoid this problem.  So, for
those it would be good if the package could specify that an
extension module is optional and that failure to build it for any reason
does not mean failure of the "build".

Note that twisted's already attempts to do this for certain of
its extension modules, and if I understand correctly it is able to
detect and handle failure to build if gcc fails, but not if gcc is absent.
msg80682 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2009-01-28 00:25
As far as Twisted is concerned, the two cases Tarek mentioned aren't
really relevant.  In order for Zooko's use case to be handled ("install
Twisted on an OLPC"), what would be ideal is if there were a way to tell
distutils about an extension which is optional.  It doesn't matter if it
can be built or not.  Any failure is irrelevant.

This doesn't necessarily have to mean that errors pass silently.  The
build failure can still be reported.  The build could even fail by
default and distutils could gain a new option for switching the behavior
to succeeding if only optional extensions fail to build.  Or it could be
the other way around, with an option to make the build fail if any
optional extensions cannot be built (this approach favors people who
don't really know what a "build" is and just want something installed,
leaving the more advanced option to people who are able to figure out
that there are options that control build behavior).
msg80896 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2009-02-01 02:20
Hey check it out -- simplejson can try to build its extension module,
and when it fails to compile (in this case because there is no
Python.h), then it prints out a warning message and finishes a
successful build:
msg80897 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2009-02-01 02:21
Oh!  And simplejson is able to cleanly fall back to pure Python when gcc
is not found, as well.  Perhaps Twisted could use simplejson's approach.
msg80898 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2009-02-01 02:27
Here's Twisted failing to build when gcc is not installed:
msg80899 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2009-02-01 02:59
Here's Twisted failing to build because Python.h isn't found:
Date User Action Args
2009-03-31 21:59:56tareksetstatus: open -> closed
resolution: duplicate
superseder: Optional extensions in
assignee: tarek
2009-02-01 02:59:09zookosetfiles: + Twisted-build-no-Python.h.txt
messages: + msg80899
2009-02-01 02:27:59zookosetfiles: + Twisted-build-with-no-gcc.txt
messages: + msg80898
2009-02-01 02:21:52zookosetfiles: + build-with-no-gcc.txt
2009-02-01 02:21:42zookosetmessages: + msg80897
2009-02-01 02:20:38zookosetfiles: + build-with-no-Python.h.txt
messages: + msg80896
2009-01-28 00:25:07exarkunsetnosy: + exarkun
messages: + msg80682
2009-01-28 00:03:16zookosetmessages: + msg80681
2009-01-04 11:43:03tareksetpriority: normal
type: enhancement
messages: + msg79047
nosy: + tarek
versions: + Python 3.1, Python 2.7
2008-12-20 14:09:52zookocreate