Author LRN
Recipients BreamoreBoy, LRN, WhiteTiger, davidfraser, giampaolo.rodola, lkcl, rpetrov, rschoon.old, tarek, zooko
Date 2010-08-03.14:21:53
SpamBayes Score 0.0
Marked as misclassified No
Message-id <1280845343.25.0.769850883753.issue3871@psf.upfronthosting.co.za>
In-reply-to
Content
Here's an alternative patch that enables Python building with MinGW/MSys
I've skimmed rpetrov's patch, and it seems that my patch is more about building Python than about distutils (i've patched distutils only as far as the compilation of Python itself (with the extensions already included in the trunk) requires.

Brief patch description - what i did, and why:
Python/import.c:
* briefly fixed an issue with mkdir() already pointed out in code comments

Python/dynload_win.c:
* renamed strcasecmp to pystrcasecmp to avoid conflicts with the same function defined in MinGW

Python/thread.c:
Python/ceval_gil.h:
* Made sure that Python won't try to include pthreads header when NT threads are available

Include/osdefs.h:
* Made sure that the PYCC_GCC won't lead Python into defining *nix path separation scheme
* Fixed redefinition of MAXPATHLEN (one of the few redefinitions i did fix, there are just too many of them)

configure.in:
* Added corret MACHDEP="win32" for MinGW
* Added lots of new AC_SUBST'ed variables, they are used later to alter Makefile and Setup functionality
  MACHDEP_PATH_SEPARATOR - to put ';' instead of ':' as path separator in machdep
  NTOBJS - extra object files to insert into makefile
  FROZEN_MAIN - controls the inclusion of Python/frozenmain.o into libpython. Why is it even there? Prevents the library from linking.
  NT_LDFLAGS - extra LDFLAGS to add on Windows
  SHAREDINSTALL - controls which sharedinstall target is built (works around MSys weirdness, see below)
  USE_*_MODULE - controls the inclusion of modules in Setup.config.in
  ms_windows - indicates that we're building on Windows through the rest of configure.in
  Special MINGW32_NT*/1.* case entry sets all of the above as well as some some #defines that traditionally are just hard-coded in PC/pyconfig.h (not all the defines from PC/pyconfig.h are here, there might be a need to add some more)
* Added MINGW32* to trigger CYGWIN* cases or added separate MINGW32* cases in most uname tests.
* Added MS_COREDLL definition (why is it separate from other Windows-specific AC_DEFINEs? I don't remember...)
* Added a separate check for CreateThread, a special variable (ac_cv_nt_threads) to indicate its success, and some code to switch on NT threads and switch off any other threads (including pthreads which ARE available on Windows)
* Added checks for some headers (pwd.h, sys/ioctl.h etc) and functions (spawn, spawnv, system etc) that are (are they?) traditionally available on *nix and are not checked for.
* Added correct DYNLOADFILE definition for MinGW
* Worked around AC_STRUCT_ST_BLOCKS weirdness by removing fileblocks.o from the list of object files (this one might not be portable, should check that it works on *nix)
* Fixed a strange bug in AC_COMPILE_IFELSE around "have_prototypes=yes" - when left with an empty argument [] it probuces broken (?) code in configure script (or maybe it's bash that is broken, i don't know)
* Fixed /dev/ptmx detection - MSys will tell you it's available, but it is only an MSys thing, Python shouldn't think it is there.
* Fixed PY_FORMAT_SIZE_T definition - AFAIK MinGW does not allow %z format in printf (in my test cases i've found that some definitions can prevent GCC from throwing an error about this format, but i couldn't reproduce this GCC behaviour in Python build system).
* Added ws2tcpip.h into socket test
* Added PC as a subdirectory

setup.py:
* Added module's location into sys.path. This is required to correctly import multiprocessing (it imports itself recursively, and since its .pyd file is in unusual location, it couldn't find itself)
* Added some config_h_vars.get() checks for some extensions (which is why some extra function/header checks were necessary in configure.in) - no setup.py won't try to build extensions that can't be built
* Added "'msys' in sys.builtin_module_names" kind of checks in some places. When successful, this check indicates that Python is built in MSys, and alters otherwise win32-oriented logic in setup.py and some other modules to adapt to MSys environment
* Changed the way MODULE_NAME is passed to sqlite3 extension (see below)
* Added extra library dependencies for multiprocessing extension
* Prevented setup.py from adding libX11 dependency to tkinter on Windows
* Added some msysize() calls (see below) to fix the paths passed to os.system() calls
* Added extra library dependencies for ctypes extension

PC/errmap.h:
* Re-generated it with new generrmap.c (see below)

PC/generrmap.c:
* Added an implementation of _dosmaperr() function, which is available in MSVC MSVCRT static library, but not in its MinGW counterpart (code is from PostgreSQL, should be compatible with Python license)

Lib/distutils/spawn.py:
* Fixed spawn_nt to use os.system() in conjunction with 'env.exe' program to call external tools - works like a charm in MSys ('env.exe' is an MSys tool, lives in /bin, and it hooks up MSys shell just as well as a call to sh.exe would). I don't know why i'm not calling sh.exe directly here. Just saw os.system()/env pair somewhere else, and it stuck.

Lib/distutils/util.py:
* Added a primitive msysize() function - an MSys path converter. Keeps MSys shell happy by avoiding paths with '\' path separators and '[A-Za-z]:' drive letters

Lib/distutils/ccompiler.py:
* Rigged ccompiler to use unixcompiler subclass for MSys. I didn't know at the time that there is the cygwincompiler, which should have been closer to MSys than unixcompiler. However from the looks of things it took only few lines to fix unixcompiler to work with MSys, so i kept it this way.

Lib/distutils/command/install.py:
* Added special installation schemes for MSys. Not sure if they are used during the installation of Python itself, but they look cool. It always bothered me that Python's layout on Windows is significantly different from the one on *nix, that breaks most autotools magic aimed at detecting/using Python. Now i have a chance to set things right.

Lib/distutils/command/build_ext.py:
* OK, this one is a real hack. There is virtually no way to find include and library paths for GCC: they USUALLY point to /mingw/include/* and /mingw/lib , but even that is configurable in MSys, and Python doesn't have direct access to that data -  asking GCC should always work, so we call GCC via os.system() and ask it to provide include and lib directories (GCC output is fed into a file, since os.system() doesn't have any piping)
* Added libpython naming scheme for Msys (similar to the one used for 'unix')

Lib/distutils/sysconfig.py:
* Added _init_msys() initialization function which combines _init_nt(), _init_posix() and some custom fixes (which come from PC/pyconfig.h for MSVC)

Lib/sysconfig.py:
* Added some more "'msys' in sys.builtin_module_names" checks to make sure that some code branches get executed or not.

Makefile.pre.in:
* Added all the variables from configure.in to insert correct object modules and compiler/linker flags
* Added explicit --export-all-symbols for building libpython
* Added some extravagant rules to build PC/generrmap.h and build exceptions.o which depends on it
* Added special msys sharedinstall variant with fixed paths (otherwise, with normal sharedinstall, some stuff ends up in $(DESTDIR)/MSys/1.0.11/usr/local or something like that due to MSys substituting /usr/local or any other prefix for its Win32 counterpart, which in this particular case is completely wrong, since prefix is always a relative path which shouldn't be fixed)
* Added some Windows-specific clean target commands

Modules/makesetup:
* Added some MinGW specific definitions (cloned from Cygwin's ones, didn't have to alter them after that)

Modules/_ctypes/callproc.c:
* Replaced some MS_WIN32 with _MSC_VER, since the code they guard works only with MSVC

Modules/_ctypes/_ctypes_test.c:
* Prevented preprocessor from inserting dllexport attributes (their presence disables automatic symbol exporting for module initialization function)

Modules/getpath.c:
* Fixed reduce() to work with paths with mixed separators and with paths with Windows-style drive letters

Modules/socketmodule.h:
* Avoided some clever MSSDK detection trap

Modules/_multiprocessing/semaphore.c:
* Fixed some #ifdef guards (the test is ambiguous: it can refer to broken sem_getvalue() - on *nix, or to the lack of one - on Windows)

Modules/Setup.dist:
* Used MACHDEP_PATH_SEPARATOR instead of hard-coding ':' separator into MACHDEPPATH
* Removed posixmodule and pwdmodule from the list of unconditionals modules, now they are conditional (see below)

Modules/_sqlite/connection.c:
Modules/_sqlite/cache.c:
Modules/_sqlite/statement.c:
Modules/_sqlite/row.c:
Modules/_sqlite/sqlitecompat.h:
Modules/_sqlite/cursor.c:
Modules/_sqlite/prepare_protocol.c:
Modules/_sqlite/module.c:
* I had some annoying problems passing MODULE_NAME=\"sqlite3\" - with escaping backslashes. It doesn't work well with my msysize(). Eventually i gave up and removed the need for doublequotes in the definitions by using stringifying feature of C preprocessor. Looks clumsy, but i do not insist on naming the macro Py_SQLITE_TO_STRING(), it can be something shorter...

Modules/python.c:
* Fixed main function prototype - MinGW doesn't work with wmain()

Modules/Setup.config.in:
* Made some modules configurable - some of them are not available on Windows, while others are only required to be built early on Windows. Adds MACHDEP_PATH_SEPARATOR definition as well.

Modules/posixmodule.c:
* Widened some include guards to include MinGW, added some includes for MinGW, disabled some structure re-definitions (it should be noted that i've used a patched version of MinGW's w32api package that already includes symlink functionality - the patch should eventually make it into official w32api release, so this particular fix in posixmodule might not be necessary for a while)

Modules/main.c:
* Fixed PATH_MAX redefinition
History
Date User Action Args
2010-08-03 14:22:24LRNsetrecipients: + LRN, lkcl, zooko, davidfraser, giampaolo.rodola, tarek, rpetrov, rschoon.old, WhiteTiger, BreamoreBoy
2010-08-03 14:22:23LRNsetmessageid: <1280845343.25.0.769850883753.issue3871@psf.upfronthosting.co.za>
2010-08-03 14:22:20LRNlinkissue3871 messages
2010-08-03 14:22:17LRNcreate