this patch uses work from #3871 to get a build of python for win32
by running msys under Wine, the windows emulator, on linux.
no proprietary operating system or proprietary software was used.

/bin/sh.exe is so xxxxing unbelievably slow on msys under wine
(each sh.exe takes two seconds to start up, loads 240 fonts
and 40 libraries) that running configure was completely intolerable.

consequently, given that PC/config.h is perfectly useable on nt,
this patch totally bypasses the majority of autoconf tests, and
thus cuts the configure time down from three hours to (only!)
fifteen minutes.

configure must be run as:
   ./configure --prefix/python25 --enable-win32build=yes  \
               --enable=shared=yes

there is an additional bug in msys /bin/sh which stops python.exe
from executing scripts that are handed to it on the command-line.
scripts that are fed as standard-input are absolutely fine
(even the same script).  running the same script from python.exe
from wineconsole cmd prompt is _also_ fine.

as a result, the build process terminates at:
   ./python.exe -E setup.py build

to complete the build, "wineconsole cmd" must be run, cd $(BUILDDIR)
and then run the commands manually:
   ./python.exe -E setup.py build
   ./python.exe -E setup.py install
then, back in the borked /bin/sh.exe, make install can be run.

stunningly, this does actually work: python.exe is build,
libpython2.5.dll and libpython2.5.dll.a are built... amazing.

other necessary workarounds:

* gcc -shared .... --out-implib=libpython2.5.dll.a .... etc fails
  spectacularly to create both the .a implib _and_ the .dll.
  consequently, it was necessary to split these up and use dlltool
  to create the dll, implib and .def file etc. etc.

* ar and ranlib on msys, versions 2.16, 2.17 _and_ 2.19 of binutils
  are borked.  2.16 and 2.17 throw an exception; 2.19 segfaults.
  consequently, building of $LIBRARY had to be terminated with
  prejudice.  --enable-shared=no will fail miserably.

regarding testing: it's all a bit... odd.

* ctypes tests cause segfaults!  the S8H one manages a segfault
  all on its own, but there are other tests that cause corruption
  that only shows up later... euurgh.

* builtin test_open fails due to putting \r\n instead of \n in the file

* test_file.py gets a sharing violation

* test_str gets a great one!

   test_str^M
   fixme:msvcrt:MSVCRT__sopen : pmode 0x01b6 ignored
   fixme:msvcrt:MSVCRT__sopen : pmode 0x01b6 ignored
   err:seh:setup_exception_record stack overflow 872 bytes in thread
001e eip 7bc3b1dc esp 00410fc8 stack 0x410000-0x411000-0x610000

   yup.... test_floatformatting.

   ahh, this is gooood :)


overall it's a damn good start - i'm dead impressed with
wine and msys in that they exist at all.

compiling and testing python under wine + msys should help
test and improve both those projects, and both this patch
and #3871 will help get python off of its dependence on
proprietary software.

Are you seriously requesting inclusion of the file f into Python? It
contains many changes that are unsuitable for inclusion, such as the
removal of graminit.c and the removal of configure.

(besides, I personally think that the ability to compile Python with
msys under Wine doesn't belong into core Python, because it is a
minority platform)

Is Msys+Mingw32 (running on a regular Windows) an interesting
configuration to support?
The build tools are similar to the ones used by cygwin, except that the
C runtime is msvcrt.

martin, hi, thanks for responding.

* graminit and configure were removed because they are built
automatically.  no project should ever include auto-generated files so i
assumed that it would be reasonable to remove them from the python_2.5.2
"original" git commit that i did, in order to produce the diff.  on a
build, graminit got _replaced_ with one that had ^M after evvvvery
siiingle line.

so - if graminit and configure _are_ in the main
python source tree, it's a serious mistake that should be
rectified _immediately_.

... but i doubt that, and so graminit and configure "appearing"
to be "removed" can be ignored.

* regarding your comment that compiling python under msys under wine
is a "minority platform", i believe that that is also a serious
assumption.

the reason why it's a minority platform is because... THERE WASN'T
ANY CHOICE.

i.e. msys and wine simply have not been good enough - and certainly
not _demonstrated_ as being "good enough" - which this patch shows
that they now most definitely are - to _have_ any choice about
whether python should be compiled with purely free software tools.

instead of depending on some xxxxing proprietary piece of double-xxxx
operating system _and_ the development IDE it walked in on.

sorry about that - just to emphasise how distasteful i find it to
be _forced_ to use proprietary software, and i'm not the only
person.

basically, it should be pretty clear that now that python _can_
be compiled for win32 using an entirely free-software platform,
the proprietary build chain should be absolutely dropped like a
red-hot stone.

... but regarding "minority platform"?  that's .... really quite
funny and ironic.

it was a _non-existent_ platform until about... yesterday :)

hi amaury, thanks for responding.

> Is Msys+Mingw32 (running on a regular Windows) an interesting
> configuration to support?

 [wine+]msys+mingw32 is used to _build_ python - not depend on it.
 [wine+]msys+mingw32 _replaces_ the proprietary build toolchain MSVC.

 clarification:

 * in the case of #3871, msys+mingw32 replaces MSVC.
 * in the case of #4954, wine+msys+mingw32 replaces MSVC _and_ windows

 but in either case, you end up with a complete build of python.exe,
 libpython2.5.dll and modules that is perfectly well capable of
 running under both wine _and_ native windows...

 ... WITHOUT requiring, in any way shape or form,
 EITHER msys OR mingw32.

 in other words, it's a big damn deal.

 no more proprietary dependence.

 ... let me put it this way, martin: if i told richard stallman that
 you said that "msys+mingw32 was a minority platform" he'd have
 a fit!! :)


> The build tools are similar to the ones used by cygwin, except that the
> C runtime is msvcrt.

 yeeees.... sort-of - but if you do s/#ifdef __CYGWIN__/#if defined
(__CYGWIN__) || defined(__MINGW32__) it all goes _horribly_ wrong :)

 you actually have to do s/#ifdef _MSC_VER/#if defined(_MSC_VER) ||
defined(__MINGW32__) because _MSC_VER is used alll over the place
 to detect and indicate a win32 build (as separate and distinct
 from a cygwin build).

updated patch - also removes quotes removal quotes of graminit and
configure so that martin is happier :)

also included is an updated version of #4956 as it's an essential
integral part of compiling and using python.exe under msys that
it actually WORK! :)

> Is Msys+Mingw32 (running on a regular Windows) an interesting
> configuration to support?

It is certainly desirable to be able to build extension modules
with this configuration; AFAIU, distutils already supports that
case. Whether or not it is desirable to be build Python from
source in this configuration, I don't know - most people that
want to build with mingw seem to be using the Python binaries
available from python.org, or from ActiveState. I'm not sure
whether mingw is capable of building Python correctly, with
assembly manifests and all.

> * graminit and configure were removed because they are built
> automatically.  no project should ever include auto-generated files so i
> assumed that it would be reasonable to remove them from the python_2.5.2
> "original" git commit that i did, in order to produce the diff.

Please trust that Python puts generated files into the repository for
good reasons. This is not going to change, regardless of how evil you
consider it.

> ... but regarding "minority platform"?  that's .... really quite
> funny and ironic.
> 
> it was a _non-existent_ platform until about... yesterday :)

Ok, so feel free to continue to work on it; good luck with
that project. If the project is still alive in a few years
from now, and enjoys a significant user base, we should reconsider
accepting patches.

>  no more proprietary dependence.

So what CRT do you link with? Is it msvcrt? Which version?

If building with mingw becomes wide-spread, care must
be taken that it uses the same CRT as the official binaries.
Otherwise, it will be a desaster for Python.

> Please trust that Python puts generated files into the repository for
> good reasons. 

 i can respect that :)  for no reason other than if somone says
 "please trust", i do :)

> So what CRT do you link with? Is it msvcrt? Which version?

i was _just_ beginning to wonder about that, after i saw
rpetrov's comments about MSCVER stuff.

http://www.mingw.org/wiki/SpecsFileHOWTO

aww, _frick_. :)

well... it's _going_ to be msvcr80.  why don't mingw ship with
a pre-prepared msvcr80 specfile??

*sigh*.

will get back to you on that one - it may make a difference on
the regression tests (250 passed, 12 failed, 8 skpped).

> It is certainly desirable to be able to build extension modules
> with this configuration;

 yeah, and the nice thing is - it works, too! :)

> AFAIU, distutils already supports that case.

 not without modification, it doesn't: #3871 adds "proper"
 mingw32 compiler detection and link options, such as support
 for dll.a which isn't auto-detected (but is on cygwin)

 etc. etc.

> Whether or not it is desirable to be build Python from
> source in this configuration, I don't know - most people that
> want to build with mingw seem to be using the Python binaries
> available from python.org, or from ActiveState.

 i tried that a few months ago.

 1) there is no .tgz for python-win32 so i was forced to
   install from a binary package

 2) there is no .exe so i was forced to install from msi,
   a proprietary installer - which as a free software
   developer, i have issues with, but i tried it anyway.

 3) msi installed, thanks to winetricks, but segfaulted
    and borked.  as i didn't really want to use it, i
    wasn't that unhappy.

 4) the msi turns out to be understood by the free software
    cabextract package.  but... the filenames are all borked
    and mangled.

 at that point, i decided i'd had enough: i wasn't going to
 go through 100 files looking for the one that _might_ be
 the python25.lib implib, plus other files that i'd need
 to do a non-proprietary-dependent build of python software.

> I'm not sure
> whether mingw is capable of building Python correctly, with
> assembly manifests and all.

 python setup.py build seems quite happy: rpetrov added .S
 to the compile-extensions and happily compiled win32.S
 in libffi with it.

 the only slight issue there is that the assembly file
 is incompatible with MSVC and so it borked (not at build
 time, unfortunately - at runtime).

 there's been a gcc bugreport raised, to get interoperability
 back (urk).

 GCC issue #36834.

yaay!  here's the regression test log, including some
loovely wine segfaults :)

summary:

250 tests OK.

12 tests failed:
    test_builtin test_cpickle test_file test_gzip test_locale
    test_mailbox test_os test_pep277 test_socket test_unicode_file
    test_xpickle test_zipfile
60 tests skipped:
 ....
 ....
 ....
8 skips unexpected on win32:
    test_bz2 test_cProfile test_bsddb test_profile test_tcl
    test_sundry test_sqlite test_winsound

not baaad :)

>  2) there is no .exe so i was forced to install from msi,
>    a proprietary installer - which as a free software
>    developer, i have issues with, but i tried it anyway.

Please understand that MSI is *not* a proprietary installer.
Instead, it is just a data format to be interpreted by the
installer. Whether that installer is proprietary or not
depends on whether you use the Microsoft one, or write one
yourself (or perhaps use a free interpreter for MSI that
somebody else might have written).

So to help Wine, it would be best to write a proper MSI
interpreter.

http://www.winehq.org/pipermail/wine-users/2006-May/021624.html

_drat_.

a rebuild of wine, adding a workaround to cope with lack of
support for msvcrt80 xml-based process files, is required,
commenting out a couple of functions from kernel32!

euuw :)

so - that's going to delay things somewhat.  i'll test with
msvcr71, anyway, for the time-being, see if that is any
better than msvcrt (default).

I'm rejecting this patch for the moment, as it seems far from being
ready yet. If you consider it complete and applicable-as-is (i.e. if you
can't think of any possible further improvement to the patch), please
submit a new issue. If I had to review the patch "x", I would reject it
for many reasons, e.g. because it is not against the trunk, and because
it removes graminit.h.

workarounds for a couple of wine bugs,
also includes e.g. #4977 64-bit assumptions on 32-bit systems.

updated. incorporating roumen's work as well.

manifests and rc files for msvcr80 build

martin, so sorry, i didn't see your comments - no dang hell no i'm
not done yet.

regarding graminit.h: graminit.h isn't being removed - i keep editing it
out of the patch.
i'm not _submitting_ it as part of the patch because what happens is that
Parser/pgen.exe generates CRLF and EVVVVERY single line of graminit.h
gets replaced.

so it's a bit irritating to keep having to even see it, so i removed
it from my local repository.

if i accidentally submit its removal please ignore it, entirely.

people _are_ tracking the progress of this development, and assisting
in the work.  i'm using the bugtracker to communicate the progress.

closing the bug looks... like... i dunno - it's not very helpful,
though, i have to say.

Hello everyone,

I'm seconding the wish to be able to build Python with MSYS on Win32,
and I can give several technical reasons:

It is my understanding that in order to build an extension for Python on
Win32, one requires to use the same compiler as the one Python has been
built with.

Therefore, the choice of compiler affects not only Win32 developers
working on the interpreter itself, but also developing and porting C/C++
based extensions.

As of now, Microsoft has released its Visual C++ compiler in various
versions, some free of cost (Express), some expensive (Studio). Express
versions change every year, while Studio versions change every two years. 

Express versions supersede each other, so once a new version is
available, the older version is virtually impossible to get.

Python binary releases for Win32 use the Visual Studio 2005 compiler,
which is not freely available, but must be bought. 

Right now, it's impossible to build an extension using distutils without
hacking the VC builder a bit. VCExpress uses a slightly different
registry layout for various settings, which causes the builder to fail.

It takes considerable effort to keep Python/Win32 setup tools compatible
for VC, and all this is done to support the native compiler
sufficiently, a goal that appears to me more and more as an aesthetic
choice rather than a practical one.

A developer wishing to deploy Win32 binaries for his extensions is
currently forced to go through above hurdles or eventually buy a
Microsoft product to make the pain go away.

Cross-building extensions for Python on Unix using Wine is impossible
because no supported version of either VS or VC++ works.

I call this a terrible situation.

Cygwin is the second alternative.

Building Python using Cygwin makes Python and all its extensions
dependent on cygwin.dll, plus it's also, as far as I gather, imitating a
Unix environment for runtimes, rather than a native Win32 one, which may
cripple an applications abilities to adapt to its target platform.

From my perspective, MSYS does deliver. Resulting binaries depend only
on available libraries coming with the OS. MSYS is practically available
to everybody. MSYS also supports cross compiling, so Python binaries
could even be built without Wine. On Windows, Eclipse CDT supports MSYS
as a native backend.

I would actually very much prefer if the default distribution of Python
for Win32 were to use MSYS, but I understand that developers working on
the Win32 platform prefer to use their native tools.

It's also unnecessary, as you can gather from the following:

I mainly develop games and multimedia applications on the Linux
platform, for convenience reasons. Using MSYS, I could easily deploy a
minimal custom-built version of Python with required extensions in one
package, without any dependency on simulated unix enviroments. 

I could cross-build these apps on Linux, and test them for Win32 within
Wine. If they work in Wine, they will definitely work on Windows.

I would need neither a VC/VS nor a Windows license to support it.

I call that a wonderful situation.


<offtopic>LKCL: although you are clearly a mad scientist, please do
continue your work. Do not get frustrated by ignorant responses and keep
calm. You do pioneering, disruptive work, and such work will always be
misunderstood and criticized because it requires a throughout
understanding of future possibilities.</offtopic>

Nevertheless, I see the feature you are trying to implement as trivial
and self-explaining, something that should have been part of Python for
a long time. People like me never took the time to file a ticket or make
a comment though. We are busy bastards, I apologize.