classification
Title: distutils2 should be able to compile an Extension based on the Python implementation
Type: enhancement Stage: resolved
Components: Distutils2 Versions: Python 3.3
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: tarek Nosy List: alexis, dholth, eric.araujo, jkloth, santoso.wijaya, tarek
Priority: normal Keywords:

Created on 2011-04-25 16:56 by dholth, last changed 2014-03-13 03:13 by eric.araujo. This issue is now closed.

Messages (9)
msg134394 - (view) Author: Daniel Holth (dholth) Date: 2011-04-25 16:56
It might be useful to be able to declare optional Extensions that for example won't attempt to compile on Jython or any implementation of Python for which the extension (a) doesn't work or (b) would be slower than the Python fallback.

I suppose this could be accomplished without changing anything by packaging the extension separately and using the extant "; plat=win32" style dependency qualifiers?
msg134474 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-04-26 16:07
The environment markers are specific to PEP 345, that is metadata, which includes dependencies, so that you can depend on something only on a given platform.  Your proposal of putting C extensions in another distribution and optionally depend on it should work right now, but is IMO too cumbersome.  Why not extend the format of the Extensions section in setup.cfg so that it supports environment markers?
msg134477 - (view) Author: Alexis Metaireau (alexis) * (Python triager) Date: 2011-04-26 16:39
This raises a concern about python specific python implementations dependencies. 

We probably could extend PEP 345 in order to support things such as 'platform.python_implementation == "cpython"'.
msg134533 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-04-27 08:28
Alexis is right, the implementation name is not currently available as an environment marker.  Maybe another approach could be used: consider all extensions optional on non-CPython platforms?
msg134562 - (view) Author: Daniel Holth (dholth) Date: 2011-04-27 13:36
Yes, putting implementation-specific environment markers in the Extensions section is the idea (instead of the haphazard and occasionally-applied 'Python code in setup.py appends to a list of extensions' method).

An Extension can only be optional when there is equivalent and working Python code. This isn't likely to happen for libraries like PIL where the foreign code provides core functionality instead of a speedup.

The reason I submitted this bug is that as Pypy's C API improves it can successfully compile Extensions to its detriment. Unlike Jython which IIRC has no C API, Pypy cannot patch Extension to always fail, but some of those Extensions are slower than the equivalent Python.

I think there ought to be a declarative way to mark an extension as

a. Required
b. Optional and preferred
c. Optional, not preferred

depending on the Python implementation. For (b) and (c) it would also be quite cool to have a standard flag or runtime mechanism to switch between the C and Python implementation to see which one was really faster (could be done with an import hook?).
msg134795 - (view) Author: Daniel Holth (dholth) Date: 2011-04-29 17:20
from docs.python.org:

platform.python_implementation()

    Returns a string identifying the Python implementation. Possible return values are: ‘CPython’, ‘IronPython’, ‘Jython’.

    New in version 2.6.

... and it seems pypy identifies itself as 'PyPy'.
msg134805 - (view) Author: Alexis Metaireau (alexis) * (Python triager) Date: 2011-04-29 18:23
On 29/04/2011 18:20, Daniel Holth wrote:
>      New in version 2.6.

Yep that's it. We would need to backport it in the python2 port of 
packaging (namely distutils2), but it would do the trick.

I just started a discussion about that on the fellowship (to know if we 
can add this to PEP 345).
-- 
Alexis
msg136769 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-24 16:41
The PEP has been edited to add python_implementation to the list of environment markers tokens.  Now the code needs a patch to allow using markers in the extensions section.
msg144459 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-23 16:54
One remark: I’m not sure distutils2 *runs* at all on other VMs.  For one example, parsing sys.version to get the Python version relies on an implementation detail of CPython.
History
Date User Action Args
2014-03-13 03:13:50eric.araujosetstatus: open -> closed
resolution: out of date
stage: resolved
2011-09-23 16:54:31eric.araujosetmessages: + msg144459
2011-05-24 16:41:29eric.araujosettitle: distutils2 should be able to compile an Extension based on the Python implementation version -> distutils2 should be able to compile an Extension based on the Python implementation
messages: + msg136769
versions: + Python 3.3, - 3rd party
2011-04-29 18:23:20alexissetmessages: + msg134805
title: distutils2 should be able to compile an Extension based on the Python implementation version -> distutils2 should be able to compile an Extension based on the Python implementation version
2011-04-29 17:20:29dholthsetmessages: + msg134795
2011-04-27 17:21:40santoso.wijayasetnosy: + santoso.wijaya
type: enhancement
2011-04-27 13:36:13dholthsetmessages: + msg134562
2011-04-27 08:28:57eric.araujosetmessages: + msg134533
versions: + 3rd party, - Python 2.7
2011-04-26 16:39:35alexissetmessages: + msg134477
2011-04-26 16:32:42jklothsetnosy: + jkloth
2011-04-26 16:07:57eric.araujosetmessages: + msg134474
2011-04-25 16:56:37dholthcreate