Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use The CygwinCCompiler Under Cygwin #46697

Closed
dstanek mannequin opened this issue Mar 21, 2008 · 15 comments
Closed

Use The CygwinCCompiler Under Cygwin #46697

dstanek mannequin opened this issue Mar 21, 2008 · 15 comments
Assignees
Labels
build The build process and cross-build stdlib Python modules in the Lib dir

Comments

@dstanek
Copy link
Mannequin

dstanek mannequin commented Mar 21, 2008

BPO 2445
Nosy @tarekziade, @merwok, @embray, @zooba, @ma8ma
PRs
  • bpo-2445: Allow the UnixCCompiler.find_library_file to correctly find DLL import libs (.dll.a) on Cygwin #4136
  • Files
  • cygwin-smaller.diff: Slimmed down Cygwin patch.
  • 3.4-distutils-shlibext.patch: using UnixCCompiler on Cygwin
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tarekziade'
    closed_at = <Date 2021-02-03.18:19:07.597>
    created_at = <Date 2008-03-21.16:41:36.688>
    labels = ['build', 'library']
    title = 'Use The CygwinCCompiler Under Cygwin'
    updated_at = <Date 2021-02-03.18:19:07.597>
    user = 'https://bugs.python.org/dstanek'

    bugs.python.org fields:

    activity = <Date 2021-02-03.18:19:07.597>
    actor = 'steve.dower'
    assignee = 'tarek'
    closed = True
    closed_date = <Date 2021-02-03.18:19:07.597>
    closer = 'steve.dower'
    components = ['Distutils']
    creation = <Date 2008-03-21.16:41:36.688>
    creator = 'dstanek'
    dependencies = []
    files = ['9804', '39141']
    hgrepos = []
    issue_num = 2445
    keywords = ['patch']
    message_count = 15.0
    messages = ['64245', '64246', '64248', '67058', '73674', '73696', '73702', '73703', '73706', '101295', '194383', '222631', '241618', '273918', '386325']
    nosy_count = 10.0
    nosy_names = ['jlt63', 'janssen', 'ocean-city', 'tarek', 'jayvee', 'eric.araujo', 'rpetrov', 'erik.bray', 'steve.dower', 'masamoto']
    pr_nums = ['4136']
    priority = 'high'
    resolution = 'out of date'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'compile error'
    url = 'https://bugs.python.org/issue2445'
    versions = ['Python 3.5', 'Python 3.6']

    @dstanek
    Copy link
    Mannequin Author

    dstanek mannequin commented Mar 21, 2008

    I was having an issue building extension with a fresh checkout of
    revision 61699. distutils was using the UnixCCompiler. This is not able
    to find thec correct libraries to link against. I made a few changes in
    this patch:

    • distutils now uses the CygwinCCompiler when building extensions on the
      Cygwin platform
    • CygwinCCompiler.static_lib_extension needed to be .lib.a instead of
      just .a
    • Added some files to the svn:ignore property on a handful of directories.

    @dstanek dstanek mannequin added stdlib Python modules in the Lib dir build The build process and cross-build labels Mar 21, 2008
    @tiran
    Copy link
    Member

    tiran commented Mar 21, 2008

    The patch contains lots of unrelated changes. Can you please provide a
    clean patch and some doc updates? The rest looks fine to me.

    @dstanek
    Copy link
    Mannequin Author

    dstanek mannequin commented Mar 21, 2008

    As Christian suggested I removed the unrelated svn:ignore changes.

    @jayvee
    Copy link
    Mannequin

    jayvee mannequin commented May 19, 2008

    The following will also let environment variables be passed to the
    CygwinCCompiler.

    --- orig/sysconfig.py	2008-05-19 00:26:03.953125000 -0700
    +++ copy/sysconfig.py	2008-05-19 00:14:51.609375000 -0700
    @@ -145,7 +145,7 @@
         Mainly needed on Unix, so we can plug in the information that
         varies across Unices and is stored in Python's Makefile.
         """
    -    if compiler.compiler_type == "unix":
    +    if compiler.compiler_type in ("unix", "cygwin"):
             (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \
                 get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
                                 'CCSHARED', 'LDSHARED', 'SO')

    @ocean-city
    Copy link
    Mannequin

    ocean-city mannequin commented Sep 23, 2008

    # I found this infomation via "svn blame"

    According to bpo-403947, cygwin once used CygwinCCompiler, and problem
    was there, swiched to UnixCCompier. (r19674)

    If this issue is not solved yet, maybe it's not good to go back to
    CygwinCCompiler. (maybe)

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Sep 24, 2008

    May be check for compiler.compiler_type (from sysconfig.py ) has to be
    replaced with a check for descendant classes of UnixCCompiler, i.e. to
    include mingw32 too ?

    Also CygwinCCompiler __init__ has to be reviewed too. As example :
    -------------
    # Hard-code GCC because that's what this is all about.
    # XXX optimization, warnings etc. should be customizable.
    self.set_executables(compiler='gcc -mcygwin -O -Wall',
    .....
    -------------
    May override in unexpected way settings from customize_compiler.

    I thin that proposed modification in sysconfig.py with removing(or
    replacing) of self.set_executables from CygwinCCompiler __init__ will
    help me for bpo-3871 (cross building python for mingw32 with distutils).

    As example I will remove a hack in the setup.py:
    -----------------------

    @@ -196,8 +196,26 @@
             if compiler is not None:
                 (ccshared,cflags) =
    sysconfig.get_config_vars('CCSHARED','CFLAGS')
                 args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags
    +
    +        # FIXME: Is next correct ?
    +        # To link modules we need LDSHARED passed to setup.py otherwise
    +        # distutils will use linker from build system if cross-compiling.
    +        linker_so = os.environ.get('LDSHARED')
    +        if linker_so is not None:
    +            args['linker_so'] = linker_so
    +
             self.compiler.set_executables(**args)

    Thanks to Hirokazu who point me, in an another thread, that cygwin build
    don't use CygwinCCompiler.

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Sep 24, 2008

    P.S. : about: static_lib_extension = ".dll.a" - it is suffix for import
    library and "unixccompiler.py.diff" patch from bpo-1706863 propose
    dylib_lib_extension = ".dll.a" .

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Sep 24, 2008

    I forgot an another issue in CygwinCCompiler __init__:
    if gcc isn't version "2.91.57" then method will set dll_libraries to
    result of get_msvcr(), but the result may be is None. In this case link
    method fail on the line libraries.extend(self.dll_libraries).

    @jlt63
    Copy link
    Mannequin

    jlt63 mannequin commented Sep 24, 2008

    I don't think the patch, cygwin-smaller.diff, is correct. The value of
    static_lib_extension needs to remain ".a". Otherwise, shared
    extensions attempting to build against a library that only offers a
    static version will fail to link.

    AFAICT, Cygwin needs a concept that is not currently supported by
    distutils -- the ability to specify a shared library *link* extension
    (i.e., ".dll.a") which is different from the shared library extension
    (i.e., ".dll"). On Unix they are the same (e.g., ".so"), but on Cygwin
    they are different because of how Windows handles DLLs -- import
    library versus DLL.

    @tarekziade tarekziade mannequin self-assigned this Feb 6, 2009
    @janssen
    Copy link
    Mannequin

    janssen mannequin commented Mar 19, 2010

    Shouldn't the import library name end with .lib? I've run up against this building PyLucene with MinGW. There seems to be code to figure this out already in CygwinCCompiler, but it's commented out (and arguably incomplete).

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Aug 4, 2013

    Hi,
    Now bpo-18654 "modernize mingw&cygwin compiler classes" contain enhancement that could be reused by this issue: patch "0007-MINGW-compiler-cygwin-provides-its-own-C-runtime.patch
    " from archive , i.e. lets avoid change in get_msvcr() that return. Mingw is also impacted but is addresses in another patches. As separate patch is 'compiler customization' - more advanced version then proposed by Jeevan Varshney (jayvee)

    A separate bpo-18634 "find import library" address .dll.a suffix based on existing distutil functionality.

    Also bpo-18633 "Mingw32CCompiler as default compiler for mingw* build" adds same fixes from unix compiler class into Cygwin one - ref '# Chop off the drive'

    So with above I think that cygwin compiler could be switched back from unix to own.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Jul 9, 2014

    From msg194383 issues bpo-18633, bpo-18634 and bpo-18654 are linked to this so is anybody up for providing a patch so that we can take this forward. See also bpo-4032.

    @ma8ma
    Copy link
    Mannequin

    ma8ma mannequin commented Apr 20, 2015

    CygwinCCompiler seems legacy code.
    In v3.4 latest rivision, Distutils has made to work that is a patching to UnixCCompiler.
    So, I have a think that doesn't need to use CygwinCCompiler to build on Cygwin.
    I have upload a patch for UnixCCompiler. The patch has modified from http://cygwin.org package.

    Other big issues and patches for build on Cygwin:
    bpo-21085, bpo-13756 -- compile error
    bpo-14598, bpo-21124 -- C extension module

    @embray
    Copy link
    Contributor

    embray commented Aug 30, 2016

    I have confirmed Masayuki's patch (3.4-distutils-shlibext.patch) fixes a few build issues on Cygwin.

    At its core, what it's fixing is allowing UnixCCompiler.find_library_file to find import libraries on Cygwin.

    That in turn is necessary for --with-system-ffi to work, which is currently needed; on Cygwin64 at least the ffi compile as part of _ctypes fails (a separate issue to be addressed separately). But in the meantime making --with-system-ffi work again goes a long way.

    It should be noted there is a patch at bpo-18654 to make the CygwinCCompiler class work again, and switch to using it to handle Cygwin-specific build issues as origially intended. I intend to review that ticket and spend some time on it, but it's more complex and not necessarily the simplest way forward.

    In the meantime this one-line patch accomplishes a fair bit.

    @zooba
    Copy link
    Member

    zooba commented Feb 3, 2021

    Distutils is now deprecated (see PEP-632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

    If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools

    @zooba zooba closed this as completed Feb 3, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    build The build process and cross-build stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants