The use of -mno-cygwin at http://bit.ly/qAIsZV causes build failures with versions of MinGW GCC. For example http://bit.ly/llK3f3

I recently ran into it trying to build a Mercurial snapshot using a MinGW GCC v4.6.2 flavor http://pastie.org/2274486

Why is -mno-cygwin still be used, and is there any reason why it can't be removed?

Thanks, Jon

I can confirm the option has been removed. For those that don't want to believe a random person's comment on a bugtracker, here's the commit:

http://repo.or.cz/w/official-gcc.git/commit/2637551aa9314c46cf4205d435ab5e8944e9ac8a

The changelog is a bit cryptic, but note the last two items "removing documentation from removed option".

Hi!  Thanks for the report.  This can’t be fixed in 2.5, 2.6 or 3.1, which are in security-fix only mode, but we can do something for the active branches.

A quick web search finds reference of this deprecation/removal as far as 2007.  Does anyone have more official documentation?  If we can find the first gcc version that deprecates this option, then I’ll be able to make a patch with a version check (in order not to change previously working code).

(In the future, it would be nice of you not to use URL obfuscators or pastebins. Such services offer no guarantee of continued availability of hosted resources or links, but it is useful to keep information on this bug tracker for future reference.  Thank you.

The first link is this: http://hg.python.org/cpython/file/4feb889d3bed/Lib/distutils/cygwinccompiler.py#l296
The second: http://stackoverflow.com/questions/6034390/compiling-with-cython-and-mingw-produces-gcc-error-unrecognized-command-line-op
The compile log from the third link is attached as a file.)

I forgot to answer this:

> Why is -mno-cygwin still be used,
Simply because distutils had no dedicated maintainer for a long time, and because nobody was aware of this change.  I don’t think any Python core developer is using Cygwin.

> and is there any reason why it can't be removed?
For distutils, we have to work on eggshells when fixing bugs to make sure we don’t break third-party code that relies on known bugs or works around them.  That’s why I propose adding a version check* instead of just removing the option.

* i.e. removing -mno-cygwin if self.gcc_version >= some version

> Does anyone have more official documentation? 

The commit I  linked to has the full option removal at October 7 2010 (see the fifth item in the changelog entry). Any GCC (major) version released after that will have it completely removed.

Deprecation happened (according to the first doc change mentioning this option as being deprecated) on 2009-03-25: http://repo.or.cz/w/official-gcc.git/commit/6609be22531f447aeddbb3f670ad7883036cb23f

It looks like GCC 4.4 and up have this change in them, by looking at their commit logs. Anyone using GCC 4.3 on MinGW should be shot (but that's just my humble opinion, rather radical I admit ;-) )

Note this has really nothing to do with current Cygwin or the cygwin platform; MinGW was part of the Cygwin source for a long time, and every MinGW compiler was burdened to search the Cygwin header and lib paths. These commits fixed that problem.

MinGW(-w64) is a native compiler for Win32 just as MSVC is. Links to msvcrt.dll and has no POSIX translation/compatibility layer like the Cygwin DLL.

May I ask for a reconsideration to commit a fix for this for Python 2.7 at least? With the version check it doesn't hurt anyone, instead it helps prevent further confusion and support requests on the MinGW side. Distutils pop up everywhere, and the projects depending on them rely on them working correctly. Thanks

I have confirmed on Win7 that the following 32bit MinGW flavors still recognize -mno-cygwin option and build without error:

(4.5.2) http://tdm-gcc.tdragon.net/download

(4.5.4) http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Automated%20Builds/ using the mingw-w32-1.0-bin_i686-mingw_20110624.zip download

...by compiling with:

[i686-w64-mingw32-]gcc -Wall -mno-cygwin -o helloworld.exe helloworld.c

...where helloworld.c was:

#include <stdio.h>
int main(int argc, char *argv[]) { printf("Hello World!\n"); return 0; }


I have not yet checked the latest mingw.org download from the following, but I expect that it also recognizes -mno-cygwin. I will check later and report back if appropriate.

(4.5.2) http://sourceforge.net/projects/mingw/files/

Given Ruben's 4.4 comment, something is odd here. Looks like some gcc source spelunking as these doc links make no mention of the removal:

http://gcc.gnu.org/gcc-4.6/changes.html
http://gcc.gnu.org/gcc-4.5/changes.html
http://gcc.gnu.org/gcc-4.4/changes.html

and the gcc manual is inconsistent. This summary mentions the option

http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Option-Summary.html#Option-Summary

but the details sections don't mention the option

http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/i386-and-x86_002d64-Windows-Options.html#i386-and-x86_002d64-Windows-Options

http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/i386-and-x86_002d64-Options.html#i386-and-x86_002d64-Options

Ah, our good friend grep.

should the question be "what's the first mingw gcc version that -mno-cygwin usage unnecessary" rather than finding the first version the option was removed?

and, does it matter whether you're building on win for win, or cross compiling for win from nix?

sadly, i don't know gcc internals but a post to the mingw and/or mingw-w64 ml's should get the right people weighing in.

btw, can distutils do a lightweight configure-like check for feature presence rather than going a gcc version check?

> May I ask for a reconsideration to commit a fix for this for Python
> 2.7 at least? With the version check it doesn't hurt anyone
There’s a misunderstanding: I explained why 2.5, 2.6 and 3.1 can’t be fixed, but if you look at the versions at the top of this page, you’ll see 2.7, 3.2 and 3.3 listed.  This will get fixed in these versions.

> Ah, our good friend grep.
Thanks for the exploration.  This looks complicated.

> should the question be "what's the first mingw gcc version that
> -mno-cygwin usage unnecessary" rather than finding the first version
> the option was removed?
If removing the option in these versions causes no change (i.e. doesn’t break any code), I agree.

> and, does it matter whether you're building on win for win, or cross
> compiling for win from nix?
I’m afraid I don’t know enough about Windows and MinGW to answer that.  If we can’t be sure about versions and consequences here, I’ll go to the MinGW ML.

> btw, can distutils do a lightweight configure-like check for feature
> presence rather than going a gcc version check?
There is a config command that’s not very used, but it would be IMO overkill to use it in the compiler code.

>> and, does it matter whether you're building on win for win, or cross
>> compiling for win from nix?
>I’m afraid I don’t know enough about Windows and MinGW to answer that.  If we can’t be sure about versions and consequences here, I’ll go to the MinGW ML.

Ruben's on it :)
http://sourceforge.net/mailarchive/message.php?msg_id=27864860

are you ok with a targeted patch similar to what's being discussed at

http://sourceforge.net/mailarchive/message.php?msg_id=27895558

assuming the regex search the output of `gcc -dumpspecs` idea is valid?

it is save to remove  -mno-cygwin from Mingw32CCompiler

The necessity of walking on eggs with the distutils codebase restrains me.  I’ve read the thread on sourceforge (thanks Ruben) but don’t have enough information yet to decide whether to do a version check, call gcc -dumpspecs or remove the option altogether.

BTW, there’s no need to use re.search if you’re not using a regular expression: “'no-cygwin' in output” will work.

shortly after opening this issue i removed -mno-cygwin from my 2.7.2 install and have had no issues on win7 32bit. but i understand you're hesitation.

regardless what you decide, please consider placing a summary note in the source comments as a safety net.

if you'd like me to try building a limited set of extensions to help with your decision, let me know.  i have the following mingw toolchains on my win7 32bit system:

* tdm gcc 4.5.2 - http://tdm-gcc.tdragon.net/
* mingw-w64 prefix-stripped gcc 4.5.4 - http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Automated%20Builds/
* Ruben's gcc 4.6.2 personal build - http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/rubenvb/4.6.2-1/

i'm not speaking for Ruben, but as he maintains https://github.com/rubenvb/MinGW-w64-build-scripts you might try cajoling him if you feel you need more test results ;)

re: re.search, understood, thanks...quick-testing.

Would it be practical to have a trivial compilation test to see if we are capable of using GCC with -mno-cygwin and if so, use it, otherwise drop off? I think GNU autotools uses a similar strategy for detecting compiler capabilities.

> Would it be practical to have a trivial compilation test to see if we
> are capable of using GCC with -mno-cygwin and if so, use it, otherwise
> drop off? I think GNU autotools uses a similar strategy for detecting
> compiler capabilities.

The config command does such things, but the compiler classes don’t.  I’d prefer something more lightweight, like a version check (see msg141231).

Just adding a data point: I use the "mingw32" compiler setting with the Cygwin GCC installation, to compile extensions for standard Windows Python (*not* Cygwin Python).  It would be good if any change doesn't break this scenario -- assuming it's not already broken in newer Python versions.

(For this scenario, AFAIK, the no-cygwin option is required, or at least I believe it was the last time I intentionally compiled something.)

I've come across this issue when trying to build extensions for Python 3.3 on Windows, needing a recent enough MinGW to provide a library for msvcr100. See issue #15315.

I have gcc 3.4.4 in my cygwin installation, and it still needs the -mno-cygwin option, else the resulting binary will link with cygwin1.dll (which is undesired).

I just installed the cygwin gcc4 package; this gives me gcc 4.5.3. In this version, -mno-cygwin is still recognized, and gives this error message:

gcc: The -mno-cygwin flag has been removed; use a mingw-targeted cross-compiler.

So it seems that removing the -mno-cygwin flag is definitely incorrect, as it will result in a cygwin binary.

I then installed the mingw-gcc-core package, which gave me the utility  i686-pc-mingw32-gcc, which indeed is able to create a (nearly) correct binary.

So I think we should check whether i686-pc-mingw32-gcc exists in the path, and if so, use it (without a -mno-cygwin flag). If it doesn't exist, we should continue to use gcc -mno-cygwin.

My remaining concern with i686-pc-mingw32-gcc is that it still links with msvcrt.dll in addition to linking with msvcrXY.dll; this is incorrect.

There is another concern that this applies to 32-bit mode only; in 64-bit mode,  i686-w64-mingw32-gcc should be used. However, this is issue 4709.

The 64-bit compiler is actually called  x86_64-w64-mingw32-gcc.

It would be great if this could be sorted out in time for Python 3.3. Otherwise I don't think we'll be able to use MinGW to build extensions in Windows. Unless there is a version of MinGW which supports the -mno-cygwin option, as well as libmsvcr100.

Checking for a compiler's file name is stupid. Native Windows gcc is just "gcc.exe", Cygwin native GCC is also "gcc". Some have a lot of toolchains in PATH at the same time. That's not the right way to handle this kind of thing.

*bump*

I just installed MinGW 2.6.2 32-bit on Windows XP. It doesn't accept -mnocygwin and there is no binary "i686-pc-mingw32-gcc" either.

It would be great if you could agree on an approach and get this fixed. This impacts a lot of users that want to build extensions on Windows. In the mean time users can find a hack to work around the issue here: https://gist.github.com/4466320

Geert: Someone will have to provide a patch, else the issue will go nowhere. I will insist that the patch continues to support "older" gcc installations (from 2012), at least if implemented as a bug fix to older Python releases.

In scope of this issue I would like to propose following patch set.
First step is remove checks for versions used in past millenium, i.e. to avoid checks for 15 year old binaries.

Next step is to propose customization for cygwin&mingw compilers.

And optionally If someone disagree options m{no-}cygwin  to be removed I would like to propose a patch '..check if cygwin/mingw... -m{no-}cygwin' to restore it for GCC before 4.6x.

2.7, 3.3 and 3.4 now have AC_CANONICAL_HOST. So the correct compiler should be picked up.  please could somebody validate this? or are explicit --host and --build options required for configure?

doko: this issue is about Windows; autoconf is not being used here (plus the "correct" compiler would be MSC).

Hi Matthias,

This issue is only for windows.
In scope autotool based builds compiler customization is used to 'transfer' some build settings (flags, options)  to distutils. This include compiler set in make macro (variable) CC.

Transfer is not complete but this is distutils issue out of scope here.

P.S.
(uploaded a patch to avoid syntax warning)

Guys, this looks really bad and inconveniences a lot of users. You install the latest MinGW and Distutils from their default location, try using them on **anything that requires compilation**, and get the cryptic gcc -mno-cygwin error (after having to edit the obscure distutils.cfg, of course).

Aren't Python / distutils supposed to be cross-platform? It's already hard enough to find distutils / pip setup instructions for Windows, shouldn't they at least **work**? After removing -mno-cygwin from cygwincompiler.py, I get another obscure -mdll error. This is ridiculous.

If you can't agree on a patch that detects both new and old compilers, can't you split cygwincompiler.py into several versions, or somehow provide separate mingw32-old and mingw32-new options?

> Dan added the comment:
>
> Guys, this looks really bad and inconveniences a lot of users. You install the latest MinGW and Distutils from their default location, try using them on **anything that requires compilation**, and get the cryptic gcc -mno-cygwin error (after having to edit the obscure distutils.cfg, of course).
>
> Aren't Python / distutils supposed to be cross-platform? It's already hard enough to find distutils / pip setup instructions for Windows, shouldn't they at least **work**? After removing -mno-cygwin from cygwincompiler.py, I get another obscure -mdll error. This is ridiculous.
Yes . This is reason to pack many changes in one archive 
"issue12641-modernize_cygwin&mingw_compilers.tar.gz 
<http://bugs.python.org/file29030/issue12641-modernize_cygwin%26mingw_compilers.tar.gz>", 
i.e. to remove all checks for tools used in previous millеnium.

My oldest compilers are :
a)
i386-mingw32msvc-gcc (GCC) 3.4.5 (mingw special)
Copyright (C) 2004 Free Software Foundation, Inc.
b)
gcc.exe (GCC) 3.4.5 (mingw-vista special r3)
Copyright (C) 2004 Free Software Foundation, Inc.

Check for -m{no-}cygwin flags is optional. I can not found reason this 
patch to be applied, as with implementation of compiler customization 
this is for developer guide.

Roumen

This also affects our software. I agree with Dan (danmbox): I don't understand; so many people depend on it and yet an out-of-the-box solution doesn't work.

I don't want to break the distutils package of our users because we use mingw. Within one Python script, I managed to fix it using this before the setup call:

if isWindows():
	"""
	Fix bug in cygwinccompiler: removed -mno-cygwin. This is fixed in cygwinccompiler_new.
	We hacked the distutils.ccompiler : def new_compiler : It uses sys.modules to fetch the compiler
	By modifying the sys.modules, we can choose our own compiler version.
	(this is a bug that's out there for quite some time)
	"""
	import cygwinccompiler_new
	import distutils.cygwinccompiler
	import sys
	sys.modules["distutils.cygwinccompiler"] = cygwinccompiler_new

..if I then later run setup(...), it will use my new cygwinccompiler_new, that has the '-mno-cygwin' line removed.

However, when you want to install new packages using pip from the command-line, I cannot find a suitable fix (except if I would replace the distutils.cygwinccompiler before pip'ing, then put it back). For afaik, distutils cannot be virtualenv'ed, right? So we cannot even fix the issue in a virtual environment.

If it is not possible to find out what version of gcc implemented it first; can't you simply use a pragmatic solution and run
"gcc -mno-cygwin": if it gives an error, then remove the option. That would need the least testing and would fix the issue.

Nice partial work-around.

I think it's quite clear that the decision makers for this bug have not been making rational decisions for a year and a half, so we can't really expect change. This being the open-source world, the only recourse is publicizing the issue (or is there some bounty system for python?). So I suggest you blog about it.

> I don't understand; so many people depend on it and yet an out-of-the-box solution doesn't work.

It is because of the combinations of some facts:
- Python contributors are volunteers;
- distutils is used a lot and quite brittle, which is a combination that makes us extremely cautious with changes;
- I am not a Windows user or developer;
- Martin knows Windows but apparently not Cygwin;
- nobody submitted patches until this month;
- nobody reviewed patches after they were attached.

You can choose to believe that we are unresponsible, stupid, etc. or accept that in an open, volunteer-based project, subtle changes in a niche part of the code are not committed right away.  Whatever we do, we get blamed for moving too slowly and for changing Python too fast (not by the same people :).

The good news is that Roumev submitted clear patches, and that Trent Nelson put together Snakebite, a set of computers for Python developers that run various OSes including Windows.  Using these machines to test the patches, we can now make progress.

Dear Eric,

I never said that anyone of these volunteers is unresponsible/stupid/whatsoever. It was also never my intention to express myself in this way, so I apologize if you felt harmed in any way. I was just suprised that this issue exists for so long. And since I only recently started developing on Windows, I was a bit knocked over by the additional effort that was needed to get things working (for which I blame Windows, not Python!). I've been developing for some time now and I understand completely that it's far from trivial; given the complexity of the software and fragility of distutils. So kudos to the developers that put their hours into this! 

Anyway, I have a workaround using some distutils manipulations now (see my previous post), which works for now, and I'm looking forward to a permanent fix!

With best regards,
Martin

That's bull, Eric. This is not about a corner case in cygwin. This is about mingw, which is the **main free software that builds executables on Windows**. You know, for when you don't want to require your users to install Visual Studio.

Additionally, both you and Matthias imposed artificial conditions that made it unlikely for patches to be created (search for "will insist").

Now, I have to agree that the larger python community (and not an under-resourced team like your good selves) should be involved in distutils (or choose and **support** a different package manager). But I'm not sure where I could file a bug for that (again, blogging may be the best choice).

[Martin]
Thanks for the message.  My previous message actually quoted Dan, not you, so no apology is necessary.


[Dan]
> That's bull, Eric.
Could you rephrase this?  English is not my native language.

> This is not about a corner case in cygwin.
Oops I confused cygwin and mingw, apologies.

> This is about mingw, which is the **main free software that builds
> executables on Windows**. You know, for when you don't want to require
> your users to install Visual Studio.
Let me state that I fully sympathize with people who use Windows and still want to use free software as much as possible, even though some version of Visual Studio is now available at no cost.  Again, the fact is that many core developers don’t use Windows, and the official CPython builds use MSVC, so from this viewpoint, MinGW is a niche platform for CPython.  It does not mean that patches are rejected, just that it’s harder for me to assess them.

> Additionally, both you and Matthias imposed artificial conditions that
> made it unlikely for patches to be created (search for "will insist").
Martin used that in a comment about backward compatibility, which we do take seriously.  Roumen’s patches don’t break backward compat if I read them correctly.

> Now, I have to agree that the larger python community (and not an
> under-resourced team like your good selves) should be involved in
> distutils
Yes, more feedback, bug reports, patches and reviews from the community would help.

> (or choose and **support** a different package manager). But I'm not
> sure where I could file a bug for that
An effort is ongoing to define the successors of distutils.  One problem is that it lacks features (like dependencies), and at the same time it tries to be too much (build, upload, install).  The community (Python core developers, packaging tools developers, users) is working on these things.

> (again, blogging may be the best choice).
If you want to get involved in the packaging discussions, you can join the distutils-sig mailing list.  If you want to push this specific issue forward, testing the patches on your system would help.  If you want to support Python on MinGW in general, you can contribute a buildbot to our continuous integration system, so that we can see when things break.

[Eric]
>> This is about mingw, which is the **main free software that builds
>> executables on Windows**. You know, for when you don't want to require
>> your users to install Visual Studio.
> Let me state that I fully sympathize with people who use Windows and
> still want to use free software as much as possible, even though some
> version of Visual Studio is now available at no cost.  Again, the
> fact is that many core developers don’t use Windows, and the official
> CPython builds use MSVC, so from this viewpoint, MinGW is a niche
> platform for CPython.  It does not mean that patches are rejected,
> just that it’s harder for me to assess them.

I'm going to advocate why mingw-based toolchains (mingw.org and mingw-w64) should be considered important platforms for CPython by giving some background info.

An important part of this issue has nothing to do with the politics of "free software" and everything to do with the technical benefits of using mingw-based toolchains on Windows. While you may be aware of the technical benefits, the following are important to me:

1a) MSVC forces you to link against a specific CRT version, and I don't get to decide which CRT version via a build setting. To the best of my knowledge the only way to link against another CRT version is to use another MSVC version. Many times it's important for all parts of a codebase to link against the same CRT version to prevent well-known cross-CRT object usage failures. While Mingw toolchains typically link to msvcrt.dll by default, they allow you to specify a CRT version via spec file tweaks.

1b) Often the most practical/stable way to build mixed codebases using CRT functionality is to link everything against msvcrt.dll. Using MSVC, the only way I'm aware of is to use old-and-slow VC6 which is not easily available. Updated and performant mingw toolchains enable one to link against msvcrt.dll and other CRT versions via spec file hackery.

2) While VC++ is a good IDE, many of us would like to simply use the command line compiler tools and not clutter our environment with a huge IDE we never use. MSFT has a poor record on this front, and after many fine years of making the command line tools available as part of the Windows SDK, they've regressed back to their lame ways in Windows 8 by removing the cmd line compiler tools from the SDK and embedding into the IDE similar to the old VC6 days. Enticing as it may be, I'll stop here rather than lurch into sailor-speak rage rant.

There are other benefits, but (1a) and (1b) are very important for many real-world usage scenarios on Windows. For example, easily building mercurial with mingw on Windows that work with the official MSVC-built Python. Extend this to other widely used Python and non-Python libraries. Extend this to those building binary libraries on Linux/OSX via mingw for their Windows users.

Also, there are other build automation awesomeness things you can do with mingw toolchains. For example, wouldn't it be great if you could download your source software, a complete mingw toolchain, and build the software on Windows in a sandbox using the mingw toolchain? You can with mingw and mingw-w64 toolchains. Nothing to pre-install and highly automated with minimal user intervention.

There's an additional aspect. Even though I don't use Windows as a development platform, I care about being able to propose Python to clients as a cross-platform technology (instead of, say, Java). Having an essential piece of Python infrastructure fail miserably on Windows impacts Python's appeal / credentials.

To clarify another point, I think that Python should pick and support an "official" package manager (be it distutils, pip or whatever) not eventually, but ASAP. For the time being it looks like distutils is a keystone, therefore it needs to work.

Eric, I'm assuming you are the key decision maker for any fix. As I want to see this issue stay razor focused, where specifically do you need community help?

Depending upon your needs, I can test on my win7 32bit notebook with official python 2.7.3. It also has arch 3.7.9 and ubuntu server 12.10 virtualbox vm's. I don't build python from source with mingw and my win8 64bit box isn't ready yet. The win7 box has mingw gcc 4.6.2 32bit and mingw-w64 32bit toolchains, both of which have msys and autohell goodies. I'm swamped these days but will carve out time from somewhere.