This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: C++ compilation support for distutils
Type: enhancement Stage: resolved
Components: Distutils Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, Christian H, Josef Kemetmüller, Michael.Crusoe, ahonkela, barcc, benjamin.peterson, bje, christian.heimes, dhduvall, drkirkby, ehvatum, eitan.adler, eric.araujo, jdemeyer, mdboom, rpetrov, ryandesign, skrah, tarek, vstinner
Priority: normal Keywords: needs review, patch

Created on 2005-06-17 12:29 by ahonkela, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
python_distutils_c++_support.patch ahonkela, 2005-06-17 12:29 Patch against CVS HEAD to add C++ compilation support to distutils.
py26_cxx_compiler.patch mdboom, 2009-11-23 18:05 Patch against release26-maint
py25_cxx_compiler.patch mdboom, 2009-11-23 18:06 Patch against release25-maint
distutils2-C++.patch Arfrever, 2010-04-03 00:58 Patch for Distutils2
python-2.7-distutils-C++.patch Arfrever, 2013-10-19 07:29
python-3.3-distutils-C++.patch Arfrever, 2013-10-19 07:29
python-3.4-distutils-C++.patch Arfrever, 2013-10-19 07:30
python-3.x-distutils-C++.patch Josef Kemetmüller, 2017-07-28 16:04
python-3.8-distutils-C++.patch veprbl, 2018-07-03 17:19
python-3.x-distutils-C++.patch marsam, 2018-12-27 05:03
Messages (35)
msg48477 - (view) Author: Antti Honkela (ahonkela) Date: 2005-06-17 12:29
Add more proper C++ compilation support to distutils by
propagating information on used language to compiler as
well as linker. The information is used by
unixccompiler, cygwinccompiler and emxccompiler to call
a C++ compiler instead of a C compiler. Options for the
call are formed by as a union of options to both C and
C++ compilers.

The old version worked by accident on some platforms
where you can compile C++ programs by calling the C
compiler (e.g. gcc), but not on others (e.g. Tru64
Unix, probably also others not using gcc).

The more compilcated handling of options than with
linker is necessary to properly support manual
overrides such as "CXX=g++ -foo".

The attached patch is against CVS HEAD, but applies
with minimal changes at least to Python 2.3.
msg59307 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-05 19:22
Isn't it already fixed? Need to check it at the upcoming bug day.
msg93211 - (view) Author: David Kirkby (drkirkby) Date: 2009-09-28 14:33
Just to add, I'd much appreciate if this bug could be squashed. I see
there is a patch, but it is against a 3-year old version, and is not a
small patch. Hence I am reluctant to try to apply it myself. 

It is causing a problem in the Sage project, as I tried to compile Sage
with Sun's compiler on Solaris, rather than gcc.
msg95629 - (view) Author: Michael Droettboom (mdboom) * Date: 2009-11-23 14:21
This is a rather serious bug for matplotlib -- since the C++ compiler is
not correctly selected, it can't be built with many Unix compilers, such
as Sun Studio.
msg95631 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2009-11-23 14:29
Has someone tried to apply it with a recent Python ?

I can work on it but I'll need some help from someone matplotlib or Sage
I guess..
msg95633 - (view) Author: Michael Droettboom (mdboom) * Date: 2009-11-23 15:43
I'm testing this against release25_maint and release26_maint as we
speak.  I'll post new patches when that's done.
msg95640 - (view) Author: Michael Droettboom (mdboom) * Date: 2009-11-23 18:05
Attached is a patch against the release26-maint branch.  This works for
me on Sun Workshop 6.

Note that on Sun, the C compiler is 'cc' and the C++ compiler is 'CC',
but distutils uses 'cc' and 'c++'.  However, the CXX environment
variable now works (after applying the patch), so this is less of an
issue, if it can be considered a bug at all.
msg95641 - (view) Author: Michael Droettboom (mdboom) * Date: 2009-11-23 18:06
Here's another patch against release25_maint for the benefit of those
still using Python 2.5.  It's different only in some whitespace changes.
msg100804 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-03-10 19:26
I'm attaching my own patch, which has the following advantages:
- Determination of target language is performed per source file instead of per list of source files passed to CCompiler.compile().
- Proper determination of linker.
- Proper support for CXXFLAGS and other variables.
msg100805 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2010-03-10 19:59
Moved to distutils2 (distutils is feature frozen now)
msg100806 - (view) Author: Michael Droettboom (mdboom) * Date: 2010-03-10 20:09
Does the reassignment only apply to the more recent more ambitious patch?  The original patch (and my forward porting of it) is purely a bugfix, not a feature freeze.  This is a showstopper bug for matplotlib on Solaris, for example, and I would hate to see its application further delayed.
msg100807 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2010-03-10 20:40
If we consider that Distutils didn't provide c++ support, and that it partially worked by accident (through gcc), I would call it a new feature. Especially since it requires a new option.

We are freezing the API so we can't add options to methods/functions. 

That's why it could be done on distutils2 instead, which will be released when 2.7 final is released (but not in the stdlib), and backward compatible with 2.4, 2.5, 2.6.

If we could do the proper job for C++ support in distutils2 that would be better. 

Notice that distutils2 is a fork of how distutils was in trunk last month, so it should be fairly simple to depend on it in your project, to benefit from new features etc. The goal is to push it back in the stdlib in 3.3. 

If you want to start a dev. branch of your project using distutils2, I am willing to help there.
msg101588 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-03-23 17:28
To have proper support for C++ linking in distutils2 at least in when used with Python 2.7, Makefile should set a variable, which will specify default C++ linker. I think that LDCXXSHARED can be used as a name for this new variable. I'm attaching the patch for configure.in and Makefile.pre.in.
msg101602 - (view) Author: Roumen Petrov (rpetrov) * Date: 2010-03-23 21:25
I could not found reasons python build to support C++ compiler as all source is C!
Another case is distutils where is possible external module to use C++ code.
msg101626 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-03-24 08:08
Python can be compiled using C++ compiler.
See http://docs.python.org/extending/extending.html#writing-extensions-in-c
msg101627 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-03-24 08:11
Also you would have to pass --with-cxx-main=<compiler> option to configure.
msg102216 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-04-03 01:58
I think python-LDCXXSHARED.patch is fine.
msg102231 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2010-04-03 08:47
LDCXXSHARED added in r79652 and r79657.

Now we can work on distutils2 side
msg115762 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-09-07 13:48
Arfrever’s patch looks good. There is no tests for detect_language now; should the patch be blocked by that?

(Note: Remember to use “hg import --user 'Arfrever Frehtes Taifersar Arahesis <preferred email>'” to give proper credit)
msg144117 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2011-09-16 09:25
Could this get some attention, please?
msg144191 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-17 15:16
Well, there’s no rush, Python 3.3b1 is not next week.

I’m willing to apply the patch to packaging and distutils2 (in a week or two when I’ve finished a massive cleanup and caught up with packaging), but I’d like tests for detect_language, and if possible tests for compiling C++ extensions/libs too.
msg158200 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2012-04-13 08:13
Ping, again. I'm sorry, I didn't write any of these patches and would not be a great fit for writing tests.
msg158223 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-04-13 18:00
I should be able to port the patch and add tests for detect_language, but I know very little about C++ and may not be able to write a full test that really compiles and checks a C++ program.  We’re having a sprint on the 21, I’ll see if I can work with another participant to do this.  A sample short C++ source file would help (just use some Python/C function and print something).
msg178660 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2012-12-31 10:10
I attach updated patches for distutils in case somebody wants to use them. (I privately update them once per week.)
msg213211 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-03-12 07:54
Now that distutils2 development is stopped and that distutils is no longer under a feature freeze, this is a reasonable thing to add to 3.5.  Thanks for your efforts Arfrever, and sorry you had to port and maintain this to so many different versions.
msg242706 - (view) Author: Erik Hvatum (ehvatum) Date: 2015-05-06 20:07
The 2012-12-31 of this patch (currently the latest) has the surprising effect of causing distutils CFLAGS to be dropped.  See for reference: https://bugs.gentoo.org/show_bug.cgi?id=548776

The issue I am experiencing is resolved by changing the section of the patch that reads:

+-        (cc, cxx, opt, cflags, ccshared, ldshared, shlib_suffix, ar, ar_flags) = \
+-            get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
+-                            'CCSHARED', 'LDSHARED', 'SHLIB_SUFFIX', 'AR', 'ARFLAGS')
++        (cc, cxx, ccshared, ldshared, ldcxxshared, shlib_suffix, ar, ar_flags) = \
++            get_config_vars('CC', 'CXX', 'CCSHARED', 'LDSHARED', 'LDCXXSHARED',
++                            'SHLIB_SUFFIX', 'AR', 'ARFLAGS')
++
++        cflags = ''
++        cxxflags = ''

to:

+-        (cc, cxx, opt, cflags, ccshared, ldshared, shlib_suffix, ar, ar_flags) = \
+-            get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
+-                            'CCSHARED', 'LDSHARED', 'SHLIB_SUFFIX', 'AR', 'ARFLAGS')
++        (cc, cxx, cflags, ccshared, ldshared, ldcxxshared, shlib_suffix, ar, ar_flags) = \
++            get_config_vars('CC', 'CXX', 'CFLAGS', 'CCSHARED', 'LDSHARED', 'LDCXXSHARED',
++                            'SHLIB_SUFFIX', 'AR', 'ARFLAGS')
++
++        cxxflags = cflags

This change causes the CFLAGS outputted by "python-config --cflags" to be used by distutils, as they are without the patch applied.  To me, losing those CFLAGS, especially when building plain C python extensions, seems like a significant regression.
msg299396 - (view) Author: Josef Kemetmüller (Josef Kemetmüller) Date: 2017-07-28 15:29
python-3.4-distutils-C++.patch with changes of Erik Hvatum applied.
msg299403 - (view) Author: Josef Kemetmüller (Josef Kemetmüller) Date: 2017-07-28 16:04
python-3.4-distutils-C++.patch with Erik Hvatum's suggestions applied.
msg314859 - (view) Author: Ben Elliston (bje) Date: 2018-04-03 04:01
Is there a simple workaround that one can put into setup.py in the meantime?  I tried using compiler.compiler.remove('-Wstrict-prototypes') to no avail.
msg337380 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-03-07 10:03
> I tried using compiler.compiler.remove('-Wstrict-prototypes') to no avail.

The -Wstrict-prototypes issue is a separate bug. It is fixed in Python >= 3.6 and there is an open backport PR for 2.7: https://github.com/python/cpython/pull/7476
msg371688 - (view) Author: Ryan Schmidt (ryandesign) Date: 2020-06-16 18:22
What needs to happen to get this 15 year old bug fixed? It prevents C++ Python modules from being compiled in situations where the user needs to supply CXXFLAGS.
msg371689 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-06-16 18:46
Please report the issue with setuptools. distutils is no longer under development. We recommend that all users use setuptools.
msg371757 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-17 16:21
> What needs to happen to get this 15 year old bug fixed? It prevents C++ Python modules from being compiled in situations where the user needs to supply CXXFLAGS.

Someone has to propose a PR, someone else to review it, and a core developer has to take the responsibility to merge that PR. So far, there is no PR created.
msg371840 - (view) Author: Ryan Schmidt (ryandesign) Date: 2020-06-18 22:12
Christian, thanks for the pointer. I think you're right, I probably am actually wanting this to be fixed in setuptools, not distutils, since setuptools is what people are using today. Since setuptools is an offshoot of distutils, I had assumed that the developers of setuptools would take ownership of any remaining distutils bugs that affected setuptools but I guess not. I looked through the setuptools issue tracker and this was the closest existing bug I could find: https://github.com/pypa/setuptools/issues/1732
msg379348 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2020-10-22 21:08
setuptools and other active build tools are the better target for this feature.
History
Date User Action Args
2022-04-11 14:56:11adminsetgithub: 42093
2020-10-22 21:08:31eric.araujosetstatus: open -> closed
messages: + msg379348

assignee: eric.araujo ->
resolution: wont fix
stage: test needed -> resolved
2020-06-18 22:12:39ryandesignsetmessages: + msg371840
2020-06-17 16:21:34vstinnersetmessages: + msg371757
2020-06-16 18:46:17christian.heimessetmessages: + msg371689
2020-06-16 18:22:27ryandesignsetnosy: + ryandesign
messages: + msg371688
2019-04-10 21:16:52vstinnersetnosy: + vstinner
2019-03-07 10:03:32jdemeyersetmessages: + msg337380
2018-12-27 05:03:52marsamsetfiles: + python-3.x-distutils-C++.patch
versions: + Python 3.7, - Python 3.5
2018-07-03 17:19:15veprblsetfiles: + python-3.8-distutils-C++.patch
2018-05-15 21:30:35eitan.adlersetnosy: + eitan.adler
2018-04-03 04:01:37bjesetnosy: + bje
messages: + msg314859
2017-08-01 18:56:27Josef Kemetmüllersetfiles: - GARBAGE
2017-07-28 17:50:23djcsetnosy: - djc
2017-07-28 16:04:54Josef Kemetmüllersetfiles: + python-3.x-distutils-C++.patch

messages: + msg299403
2017-07-28 15:29:27Josef Kemetmüllersetfiles: + GARBAGE
nosy: + Josef Kemetmüller
messages: + msg299396

2016-07-09 14:26:24Christian Hsetnosy: + Christian H
2016-05-26 17:13:57barccsetnosy: + barcc
2016-05-17 12:37:05skrahsetnosy: + skrah

versions: + Python 3.5, Python 3.6, - Python 3.4
2015-05-06 20:21:41brian.curtinsetnosy: - brian.curtin
2015-05-06 20:07:55ehvatumsetnosy: + ehvatum

messages: + msg242706
versions: + Python 3.4, - Python 3.5
2014-03-12 07:54:25eric.araujosetkeywords: - easy

messages: + msg213211
components: + Distutils, - Distutils2
versions: + Python 3.5, - 3rd party, Python 3.3
2014-02-26 22:47:05Michael.Crusoesetnosy: + Michael.Crusoe
2013-10-19 07:30:32Arfreversetfiles: - python-3.4-distutils-C++.patch
2013-10-19 07:30:29Arfreversetfiles: - python-3.3-distutils-C++.patch
2013-10-19 07:30:27Arfreversetfiles: - python-3.2-distutils-C++.patch
2013-10-19 07:30:24Arfreversetfiles: - python-2.7-distutils-C++.patch
2013-10-19 07:30:13Arfreversetfiles: + python-3.4-distutils-C++.patch
2013-10-19 07:29:58Arfreversetfiles: + python-3.3-distutils-C++.patch
2013-10-19 07:29:37Arfreversetfiles: + python-2.7-distutils-C++.patch
2013-04-03 23:05:15dhduvallsetnosy: + dhduvall
2013-03-14 12:46:03jdemeyersetnosy: + jdemeyer
2012-12-31 10:11:47Arfreversetfiles: + python-3.4-distutils-C++.patch
2012-12-31 10:11:36Arfreversetfiles: + python-3.3-distutils-C++.patch
2012-12-31 10:11:26Arfreversetfiles: + python-3.2-distutils-C++.patch
2012-12-31 10:11:09Arfreversetfiles: + python-2.7-distutils-C++.patch
keywords: + patch
2012-12-31 10:10:32Arfreversetmessages: + msg178660
2012-12-31 10:08:44Arfreversetfiles: - python-LDCXXSHARED.patch
2012-04-13 18:00:00eric.araujosetkeywords: + easy, - patch

messages: + msg158223
stage: patch review -> test needed
2012-04-13 08:13:57djcsetmessages: + msg158200
2011-09-17 15:16:45eric.araujosetassignee: tarek -> eric.araujo
messages: + msg144191
versions: + 3rd party
2011-09-16 09:25:12djcsetmessages: + msg144117
2011-06-14 15:35:46eric.araujosettype: compile error -> enhancement
versions: + Python 3.3, - 3rd party
2010-10-28 13:59:00djcsetnosy: + djc
2010-09-29 23:59:46eric.araujosetversions: + 3rd party, - Python 2.6, Python 2.5, Python 3.1, Python 2.7, Python 3.2
2010-09-07 13:48:06eric.araujosetmessages: + msg115762
2010-09-07 13:32:50eric.araujolinkissue9031 superseder
2010-09-06 01:12:05eric.araujosetnosy: + eric.araujo

versions: + Python 2.6, Python 2.5, Python 3.1, - Python 3.3
2010-04-03 13:51:46Arfreversetfiles: - python-distutils-C++.patch
2010-04-03 08:47:27tareksetpriority: high -> normal

messages: + msg102231
versions: + Python 3.3
2010-04-03 01:58:35benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg102216
2010-04-03 00:58:39Arfreversetfiles: + distutils2-C++.patch
2010-03-24 08:11:54Arfreversetmessages: + msg101627
2010-03-24 08:08:48Arfreversetmessages: + msg101626
2010-03-23 21:25:52rpetrovsetnosy: + rpetrov
messages: + msg101602
2010-03-23 17:28:58Arfreversetfiles: + python-LDCXXSHARED.patch

messages: + msg101588
2010-03-23 17:20:17tareksetpriority: normal -> high
2010-03-10 20:40:27tareksetmessages: + msg100807
2010-03-10 20:09:45mdboomsetmessages: + msg100806
2010-03-10 19:59:25tareksetnosy: mdboom, christian.heimes, ahonkela, tarek, Arfrever, brian.curtin, drkirkby
messages: + msg100805
components: + Distutils2, - Distutils
2010-03-10 19:26:50Arfreversetfiles: + python-distutils-C++.patch
nosy: + Arfrever
messages: + msg100804

2010-01-13 03:01:39brian.curtinsetkeywords: + needs review
stage: patch review
2009-11-23 18:31:27brian.curtinsetnosy: + brian.curtin
2009-11-23 18:06:49mdboomsetfiles: + py25_cxx_compiler.patch

messages: + msg95641
2009-11-23 18:05:27mdboomsetfiles: + py26_cxx_compiler.patch

messages: + msg95640
2009-11-23 15:43:34mdboomsetmessages: + msg95633
2009-11-23 14:29:37tareksetassignee: tarek
messages: + msg95631
2009-11-23 14:21:38mdboomsetnosy: + mdboom
messages: + msg95629
2009-09-28 14:43:48tareksetversions: + Python 2.7, Python 3.2, - Python 2.6
2009-09-28 14:33:10drkirkbysettype: compile error

messages: + msg93211
nosy: + drkirkby
2009-02-09 15:30:48akitadasetnosy: + tarek
2008-01-05 19:22:14christian.heimessetnosy: + christian.heimes
messages: + msg59307
versions: + Python 2.6
2005-06-17 12:29:06ahonkelacreate