classification
Title: Use The CygwinCCompiler Under Cygwin
Type: compile error Stage: commit review
Components: Distutils Versions: Python 3.6, Python 3.5
process
Status: open Resolution: accepted
Dependencies: Superseder:
Assigned To: tarek Nosy List: erik.bray, janssen, jayvee, jlt63, masamoto, merwok, ocean-city, rpetrov, tarek
Priority: high Keywords: patch

Created on 2008-03-21 16:41 by dstanek, last changed 2016-08-30 17:23 by BreamoreBoy.

Files
File name Uploaded Description Edit
cygwin-smaller.diff dstanek, 2008-03-21 18:17 Slimmed down Cygwin patch. review
3.4-distutils-shlibext.patch masamoto, 2015-04-20 06:57 using UnixCCompiler on Cygwin review
Messages (14)
msg64245 - (view) Author: David Stanek (dstanek) Date: 2008-03-21 16:41
I was having an issue building extension with a fresh checkout of
revision 61699. distutils was using the UnixCCompiler. This is not able
to find thec correct libraries to link against. I made a few changes in
this patch:

* distutils now uses the CygwinCCompiler when building extensions on the
Cygwin platform
* CygwinCCompiler.static_lib_extension needed to be .lib.a instead of
just .a
* Added some files to the svn:ignore property on a handful of directories.
msg64246 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-21 17:25
The patch contains lots of unrelated changes. Can you please provide a
clean patch and some doc updates? The rest looks fine to me.
msg64248 - (view) Author: David Stanek (dstanek) Date: 2008-03-21 18:17
As Christian suggested I removed the unrelated svn:ignore changes.
msg67058 - (view) Author: Jeevan Varshney (jayvee) Date: 2008-05-19 07:48
The following will also let environment variables be passed to the
CygwinCCompiler.

--- orig/sysconfig.py	2008-05-19 00:26:03.953125000 -0700
+++ copy/sysconfig.py	2008-05-19 00:14:51.609375000 -0700
@@ -145,7 +145,7 @@
     Mainly needed on Unix, so we can plug in the information that
     varies across Unices and is stored in Python's Makefile.
     """
-    if compiler.compiler_type == "unix":
+    if compiler.compiler_type in ("unix", "cygwin"):
         (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \
             get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
                             'CCSHARED', 'LDSHARED', 'SO')
msg73674 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2008-09-23 22:35
# I found this infomation via "svn blame"

According to issue403947, cygwin once used CygwinCCompiler, and problem
was there, swiched to UnixCCompier. (r19674)

If this issue is not solved yet, maybe it's not good to go back to
CygwinCCompiler. (maybe)
msg73696 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-09-24 09:52
May be check for compiler.compiler_type (from sysconfig.py ) has to be
replaced with a check for descendant classes of UnixCCompiler, i.e. to
include mingw32 too ?

Also CygwinCCompiler __init__ has to be reviewed too. As example :
-------------
        # Hard-code GCC because that's what this is all about.
        # XXX optimization, warnings etc. should be customizable.
        self.set_executables(compiler='gcc -mcygwin -O -Wall',
        .....
-------------
May override in unexpected way settings from customize_compiler.

I thin that proposed modification in sysconfig.py with removing(or
replacing)  of self.set_executables from CygwinCCompiler __init__  will
help me for issue3871 (cross building python for mingw32 with distutils). 

As example I will remove a hack in the setup.py:
-----------------------
@@ -196,8 +196,26 @@
         if compiler is not None:
             (ccshared,cflags) =
sysconfig.get_config_vars('CCSHARED','CFLAGS')
             args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags
+
+        # FIXME: Is next correct ?
+        # To link modules we need LDSHARED passed to setup.py otherwise
+        # distutils will use linker from build system if cross-compiling.
+        linker_so = os.environ.get('LDSHARED')
+        if linker_so is not None:
+            args['linker_so'] = linker_so
+
         self.compiler.set_executables(**args)
-----------------------
Thanks to Hirokazu who point me, in an another thread, that cygwin build
don't use CygwinCCompiler.
msg73702 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-09-24 10:14
P.S. : about: static_lib_extension = ".dll.a" - it is suffix for import
library  and "unixccompiler.py.diff" patch from issue1706863 propose 
dylib_lib_extension = ".dll.a" .
msg73703 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-09-24 10:24
I forgot  an another issue in CygwinCCompiler __init__:
if gcc isn't version "2.91.57" then method will set dll_libraries to
result of get_msvcr(), but the result may be is None. In this case link
method fail on the line libraries.extend(self.dll_libraries).
msg73706 - (view) Author: Jason Tishler (jlt63) * Date: 2008-09-24 11:54
I don't think the patch, cygwin-smaller.diff, is correct. The value of 
static_lib_extension needs to remain ".a". Otherwise, shared 
extensions attempting to build against a library that only offers a 
static version will fail to link.

AFAICT, Cygwin needs a concept that is not currently supported by 
distutils -- the ability to specify a shared library *link* extension 
(i.e., ".dll.a") which is different from the shared library extension 
(i.e., ".dll"). On Unix they are the same (e.g., ".so"), but on Cygwin 
they are different because of how Windows handles DLLs -- import 
library versus DLL.
msg101295 - (view) Author: Bill Janssen (janssen) * (Python committer) Date: 2010-03-19 01:25
Shouldn't the import library name end with .lib?  I've run up against this building PyLucene with MinGW.  There seems to be code to figure this out already in CygwinCCompiler, but it's commented out (and arguably incomplete).
msg194383 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-08-04 15:20
Hi, 
Now issue18654 "modernize mingw&cygwin compiler classes" contain enhancement that could be reused by this issue: patch "0007-MINGW-compiler-cygwin-provides-its-own-C-runtime.patch
" from archive , i.e. lets avoid change in get_msvcr() that return. Mingw is also impacted but is addresses in another patches.  As separate patch is 'compiler customization' -  more advanced version then proposed by Jeevan Varshney (jayvee)

A separate issue18634 "find import library" address .dll.a suffix based on existing distutil functionality.

Also issue18633 "Mingw32CCompiler as default compiler for mingw* build" adds same fixes from unix compiler class into Cygwin one - ref '# Chop off the drive'

So with above I think that cygwin compiler could be switched back from unix to own.
msg222631 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-09 19:33
From msg194383 issues #18633, #18634 and #18654 are linked to this so is anybody up for providing a patch so that we can take this forward.  See also #4032.
msg241618 - (view) Author: Masayuki Yamamoto (masamoto) * Date: 2015-04-20 06:57
CygwinCCompiler seems legacy code.
In v3.4 latest rivision, Distutils has made to work that is a patching to UnixCCompiler.
So, I have a think that doesn't need to use CygwinCCompiler to build on Cygwin.
I have upload a patch for UnixCCompiler.  The patch has modified from http://cygwin.org package.

Other big issues and patches for build on Cygwin:
#21085, #13756 -- compile error
#14598, #21124 -- C extension module
msg273918 - (view) Author: Erik Bray (erik.bray) * Date: 2016-08-30 15:05
I have confirmed Masayuki's patch (3.4-distutils-shlibext.patch) fixes a few build issues on Cygwin.

At its core, what it's fixing is allowing UnixCCompiler.find_library_file to find import libraries on Cygwin.  

That in turn is necessary for --with-system-ffi to work, which is currently needed; on Cygwin64 at least the ffi compile as part of _ctypes fails (a separate issue to be addressed separately).  But in the meantime making --with-system-ffi work again goes a long way.

It should be noted there is a patch at #18654 to make the CygwinCCompiler class work again, and switch to using it to handle Cygwin-specific build issues as origially intended.  I intend to review that ticket and spend some time on it, but it's more complex and not necessarily the simplest way forward.

In the meantime this one-line patch accomplishes a fair bit.
History
Date User Action Args
2016-08-30 17:23:45BreamoreBoysetnosy: - BreamoreBoy
2016-08-30 16:47:33dstaneksetnosy: - dstanek
2016-08-30 15:22:11christian.heimessetnosy: - christian.heimes
2016-08-30 15:05:23erik.braysetversions: + Python 3.6, - Python 2.7, Python 3.4
nosy: + erik.bray

messages: + msg273918

stage: patch review -> commit review
2015-04-20 06:57:07masamotosetfiles: + 3.4-distutils-shlibext.patch
nosy: + masamoto
messages: + msg241618

2014-08-30 03:50:09terry.reedysetstage: patch review
versions: + Python 3.4, Python 3.5, - Python 3.1, Python 3.2
2014-07-09 19:33:44BreamoreBoysetnosy: + BreamoreBoy
messages: + msg222631
2013-08-04 15:20:55rpetrovsetmessages: + msg194383
2010-10-22 10:51:54ocean-citylinkissue4032 dependencies
2010-08-21 21:40:34merwoksetnosy: + merwok

versions: + Python 3.2
2010-03-19 01:25:31janssensetnosy: + janssen
messages: + msg101295
2009-02-06 09:12:54tareksetassignee: tarek
versions: + Python 3.1, Python 2.7, - Python 2.6, Python 3.0
nosy: + tarek
2008-09-24 11:54:09jlt63setnosy: + jlt63
messages: + msg73706
2008-09-24 10:24:20rpetrovsetmessages: + msg73703
2008-09-24 10:14:52rpetrovsetmessages: + msg73702
2008-09-24 09:52:49rpetrovsetnosy: + rpetrov
messages: + msg73696
2008-09-23 22:35:33ocean-citysetnosy: + ocean-city
messages: + msg73674
2008-05-19 07:48:42jayveesetnosy: + jayvee
messages: + msg67058
2008-03-21 18:17:32dstaneksetfiles: - cygwin.diff
2008-03-21 18:17:15dstaneksetfiles: + cygwin-smaller.diff
messages: + msg64248
2008-03-21 17:25:56christian.heimessetpriority: high
resolution: accepted
messages: + msg64246
nosy: + christian.heimes
versions: + Python 3.0
2008-03-21 16:42:57dstaneksettype: compile error
2008-03-21 16:41:36dstanekcreate