classification
Title: Python build should check CPATH, C_INCLUDE_PATH for module dependencies
Type: enhancement Stage:
Components: Build Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: JanKanis, ned.deily, ronaldoussoren, yesimon
Priority: normal Keywords:

Created on 2014-05-24 21:29 by JanKanis, last changed 2015-04-03 18:23 by yesimon.

Messages (4)
msg219056 - (view) Author: Jan Kanis (JanKanis) Date: 2014-05-24 21:29
When building, pythons setup.py tries to find external sources for optional modules such as ssl, sqlite, etc. For that it searches the CFLAGS environment variable for -I options. C compilers such as gcc and clang also interpret CPATH and C_INCLUDE_PATH as extra search paths for headers/sources. Setup.py should do the same and also look in paths specified in those environment variables.
msg219071 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-05-25 03:44
setup.py already does that but only for builds on OS X (darwin); see http://hg.python.org/cpython/file/default/setup.py#l539.  The comments there suggest to me that the code was made conditional to OS X to avoid introducing regressions in builds on other platforms.  The restriction could be removed but probably only for a feature release (e.g. 3.5).  Another approach that has been suggested is to add either configure options (like in Issue21541) or environment variables (Issue5575).  In theory, the best approach might be to use the customization features provided by Modules/Setup*; unfortunately, at the moment there are some major problems with that in current Python 3.4 releases (see Issue17095 for a proposed patch for 3.4).
msg219384 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2014-05-30 11:14
I'm pretty sure that I wrote the code Ned refers to, and that's indeed only targeting darwin to avoid breaking other platforms. That code could easily be made actively globally though, the only reason I didn't do so at the time is we were still getting used to the fact that MacOS used the same build infrastructure as other major platforms and hence were overly cautious.

Note that this doesn't do the same thing as Jan requests: GCC[1] and clang[2] can add additional directories to their header file search path using environment variables. 

For Python's setup.py file the following are important:

* C_INCLUDE_PATH: a list of directories that are handled as if they are present at the end of the list of -isystem options (that is, system include files at a lower priority that the user provided ones)

* CPATH: simular, but for the '-I' option

* LIBRARY_PATH: similar, but for the '-L' option

All of them have a syntax similar to $PATH.

A patch to add support for these variables (and enables the handling of -I and -L for other platforms than darwin) should be easy enough, but I agree with Ned that this would be a new feature because it could break existing build systems (that is, building Python with this patch could result in a different build than without the patch due to the build picking up more or different external libraries).

Jan: are you willing to write such a patch? And if so, are you willing to sign a contributor agreement?

[1]: http://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html
[2]: http://clang.llvm.org/doxygen/Tools_8cpp_source.html
msg222163 - (view) Author: Jan Kanis (JanKanis) Date: 2014-07-03 08:47
I can write a patch. I haven't signed a contributor agreement but I have no problem doing so. I am not sure when I will have time to write a patch though, so it could take some time.
History
Date User Action Args
2015-04-03 18:23:18yesimonsetnosy: + yesimon
2015-04-03 15:01:40ned.deilylinkissue23858 superseder
2014-07-03 08:47:21JanKanissetmessages: + msg222163
2014-05-30 11:14:03ronaldoussorensetnosy: + ronaldoussoren
messages: + msg219384
2014-05-25 03:44:59ned.deilysetversions: - Python 3.1, Python 2.7, Python 3.2, Python 3.3, Python 3.4
nosy: + ned.deily

messages: + msg219071

type: behavior -> enhancement
2014-05-24 21:29:38JanKaniscreate