classification
Title: Cross compiling patches for MINGW
Type: enhancement Stage: patch review
Components: Build, Distutils, Distutils2 Versions: Python 3.3
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: tarek Nosy List: BreamoreBoy, alexis, amaury.forgeotdarc, cfriedt, eric.araujo, hanwen, janneke-sf, kumba, lkcl, loewis, n03702, nzjrs, rmt38, rpetrov, rwmjones, scott.tsai, sraue, tarek, wrobell
Priority: normal Keywords: patch

Created on 2006-11-16 16:57 by hanwen, last changed 2013-02-02 20:00 by hanwen. This issue is now closed.

Files
File name Uploaded Description Edit
xcompile-patch.tar hanwen, 2006-11-16 16:57 xcompiling to mingw patchbomb.
cross.patch hanwen, 2006-12-09 23:48
cross-55748.patch hanwen, 2007-06-03 23:20 patch against SVN r55748
cross-2.5.1.patch scott.tsai, 2007-10-27 06:55
cross-2.5.1.patch scott.tsai, 2007-10-27 11:07
cross-2.5.1.patch scott.tsai, 2007-10-27 11:11
cross-2.6-0.7.diff sraue, 2008-11-21 12:31
cross-3.0-0.7.diff n03702, 2008-12-06 17:17 patch against 3.0 final
Messages (56)
msg51349 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2006-11-16 16:57
Hello, 

attached tarbal is a patch bomb of 32 patches against python 2.5, that we lilypond developers use for crosscompiling python.  The patches were originally written by Jan Nieuwenhuizen, my codeveloper.

These patches have been tested with Linux/x86, linux/x64 and macos 10.3  as build host and   linux-{ppc,x86,x86_64}, freebsd, mingw as target platform. All packages at  lilypond.org/install/ except for darwin contain the x-compiled python.

Each patch is prefixed with a small comment, but for reference, I include a snippet from the readme.

It would be nice if at least some of the patches were included. In particular, I think that X-compiling is a common request, so it warrants inclusion.

Basically, what we do is override autoconf and Makefile settings through setting enviroment variables.

**README section** 


Cross Compiling
---------------

Python can be cross compiled by supplying different --build and --host
parameters to configure.  Python is compiled on the "build" system and
executed on the "host" system.  Cross compiling python requires a
native Python on the build host, and a natively compiled tool `Pgen'.

Before cross compiling, Python must first be compiled and installed on
the build host.  The configure script will use `cc' and `python', or
environment variables CC_FOR_BUILD or PYTHON_FOR_BUILD, eg:

   CC_FOR_BUILD=gcc-3.3 \
   PYTHON_FOR_BUILD=python2.4 \
   .../configure --build=i686-linux --host=i586-mingw32

Cross compiling has been tested under linux, mileage may vary for
other platforms.

A few reminders on using configure to cross compile:
- Cross compile tools must be in PATH,
- Cross compile tools must be prefixed with the host type
  (ie i586-mingw32-gcc, i586-mingw32-ranlib, ...),
- CC, CXX, AR, and RANLIB must be undefined when running configure,
  they will be auto-detected.

If you need a cross compiler, Debian ships several several (eg: avr,
m68hc1x, mingw32), while dpkg-cross easily creates others.  Otherwise,
check out Dan Kegel's crosstool: http://www.kegel.com/crosstool .

msg51350 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-11-16 21:47
Would you and Jan Nieuwenhuizen be willing to sign the contributor agreement, at

http://www.python.org/psf/contrib.html

I haven't reviewed the patch yet; if they can be integrated, that will only happen in the trunk (i.e. not for 2.5.x).
msg51351 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2006-11-17 00:32
I don't mind, and I expect Jan won't have a problem either.

What's the procedure: do we send the disclaimer first, or do you do the review, or does everything happen in parallel?

 
msg51352 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2006-11-17 00:33
note that not all of the patch needs to go in its current form. In particular, setup.py should be
much more clever to look into build-root for finding libs and include files.
 
msg51353 - (view) Author: Jan Nieuwenhuizen (janneke-sf) * Date: 2006-11-17 19:57
I do not mind either.  I've just signed and faxed contrib-form.html.
msg51354 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2006-11-25 15:12
I've sent the agreement by snailmail.
msg51355 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-12-06 20:06
I'll add my comments as I go through the patches.

cab1e7d1e54d14a8aab52f0c3b3073c93f75d4fc:
- why is there now a mingw32msvc2 platform? If the target is mingw (rather than Cygwin), I'd expect that the target is just Win32/Windows, and that all symbolic constants provided be usable across all Win32 Pythons.
- why is h2py run for /usr/include/netinet/in.h? Shouldn't it operate on a target header file?
- please include any plat-* files that you generate in the patch.
- why do you need dl_nt.c in Modules? Please make it use the one from PC (consider updating the comment about
calling initall)

b52dbbbbc3adece61496b161d8c22599caae2311
- please combine all patches adding support for __MINGW32__ into a single one. Why is anything needed here at all? I thought Python compiles already with mingw32 (on Windows)?
- what is the exclusion of freezing for?

059af829d362b10bb5921367c93a56dbb51ef31b
- Why are you taking timeval from winsock2.h? It should come from sys/time.h, and does in my copy of Debian mingw32-runtime.

6a742fb15b28564f9a1bc916c76a28dc672a9b2c
- Why are these changes needed? It's Windows, and that is already supported.

a838b4780998ef98ae4880c3916274d45b661c82
- Why doesn't that already work on Windows+cygwin+mingw32?

f452fe4b95085d8c1ba838bf302a6a48df3c1d31
- I think this should target msvcr71.dll, not msvcrt.dll

Please also combine the cross-compilation patches into a single one.
- there is no need to provide pyconfig.h.in changes; I'll regenerate that, anyway.


9c022e407c366c9f175e9168542ccc76eae9e3f0
- please integrate those into the large AC_CHECK_FUNCS that already exists


540684d696df6057ee2c9c4e13e33fe450605ffa
- Why are you stripping -Wl?

64f5018e975419b2d37c39f457c8732def3288df
- Try getting SO from the Makefile, not from the environment (I assume this is also meant to support true distutils packages some day).

7a4e50fb1cf5ff3481aaf7515a784621cbbdac6c
- again: what is the "mingw" platform?

7d3a45788a0d83608d10e5c0a34f08b426d62e92
- is this really necessary? I suggest to drop it

23a2dd14933a2aee69f7cdc9f838e4b9c26c1eea
- don't include bits/time.h; it's not meant for direct inclusion

6689ca9dea07afbe8a77b7787a5c4e1642f803a1
- what's a .x file?
msg51356 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-12-06 20:12
One more note: it would be best if the patches were against the subversion trunk. They won't be included in the 2.5 maintenance branch (as they are a new feature), so they need to be ported to the trunk, anyway.
msg51357 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2006-12-09 23:48
With cross.patch I've been able to build a working freebsd
python on linux.

Since you had little problems with the X-compile patches, I'm
resubmitting those first.  I'd like to give our (admittedly: oddball)
mingw version another go when the X-compile patches are in python SVN.

Regarding your comments:

* what would be a better to import the SO setting?

the most reliable way to get something out of a makefile into python is

  VAR=foo
  export VAR
  .. 
  os.environ['VAR']

this doesn't introduce any fragility in parsing/expanding/(un)quoting, so it's
actually pretty good.

Right now, I'm overriding sysconfig wholesale in setup.py with a

  sysconfig._config_vars.update (os.environ)

but I'm not sure that this affects the settings in build_ext.py.
A freebsd -> linux compile does not touch that code, so
if you dislike it, we can leave it out.

* I've documented the .x  extension

File Added: cross.patch
msg51358 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2006-12-09 23:50
this is a patch against a SVN checkout of last week.
msg51359 - (view) Author: Richard Tew (rmt38) * (Python committer) Date: 2007-01-07 01:50
This:

AC_CHECK_FILE(/dev/ptmx, AC_DEFINE(HAVE_DEV_PTMX, 1, [Define if we have /dev/ptmx.]))

Is being translated into:

echo "$as_me:$LINENO: checking for /dev/ptmx" >&5
echo $ECHO_N "checking for /dev/ptmx... $ECHO_C" >&6
if test "${ac_cv_file__dev_ptmx+set}" = set; then
  echo $ECHO_N "(cached) $ECHO_C" >&6
else
  test "$cross_compiling" = yes &&
  { { echo "$as_me:$LINENO: error: cannot check for file existence when cross compiling" >&5
echo "$as_me: error: cannot check for file existence when cross compiling" >&2;}
   { (exit 1); exit 1; }; }
if test -r "/dev/ptmx"; then
  ac_cv_file__dev_ptmx=yes
else
  ac_cv_file__dev_ptmx=no
fi
fi

Which exits when I do:

$ export CC_FOR_BUILD=gcc
$ sh configure --host=arm-eabi

With an error like:

checking for /dev/ptmx... configure: error: cannot check for file existence when cross compiling

I am using the latest version of msys/mingw with devkitarm to cross compile.  Is this supposed to happen?
msg51360 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2007-01-07 02:37
"checking for /dev/ptmx... configure: error: cannot check for file
existence when cross compiling"

You need to set up a config.cache file that contains 
the correct entry for 
  
  ac_cv_file__dev_ptmx 
msg51361 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2007-01-07 02:37
"checking for /dev/ptmx... configure: error: cannot check for file
existence when cross compiling"

You need to set up a config.cache file that contains 
the correct entry for 
  
  ac_cv_file__dev_ptmx 
msg51362 - (view) Author: Richard Tew (rmt38) * (Python committer) Date: 2007-01-07 22:03
config.cache is not generated or used on my Windows installation of MinGW unless --config-cache is also given as argument to configure, and from the autoconf documentation this seems to be the default behaviour.  So you might want to amend the instructions to take that into account.

Isn't requiring the user to manually create and edit config.cache resulting in unnecessary work and confusion for the them when it can be addressed in configure.in?  Given that checking files is an operation which does not work when cross_compiling is set and checking them results in configure exiting because of this, configure.in can check cross_compiling before trying these checks and avoid them allowing configure to complete.
msg51363 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2007-01-08 07:59
Regarding --config-cache, yes you're correct. 

Regarding extending configure.in, it does already say
  
"configure: error: cannot check for file
existence when cross compiling"

and exit.

What more would you like it to do?  I could add a check
that the --config-cache is given, although that is not strictly 
necessary (You can also set the variables in the environment.)
msg51364 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-03-07 12:25
A few more comments:
1. Specifying --build should not be necessary, as it should default to config.guess, right? If so, we should document that you cross-compile by passing --host. OTOH, I see it is a bug that you cannot just specify --host...
2. what are the implications of AC_CHECK_TOOLS wrt. the current AC_CHECK_PROGS invocations? There is a lot of logic to determine the C++ compiler...
3. should we include config.sub? Can we share it easily with the libffi one? Where do I get the most recent version of it?
4. Where does the mapping of system names from -dumpmachine come from? What would need to be done to eliminate this altogether? What about ac_sys_release?
5. Isn't there some autoconf way for detecting a C compiler for the build system? It shouldn't default to 'cc'.
6. I don't think the "skipping import check" warning is needed. Just silently don't perform this check.
7. What is the meaning of CROSS_TARGET? In some place, it is used like sys.platform (so it should take one of the possible values for sys.platform), in configure.in, it is set to ac_sys_system.  I think you should just use MACHDEP here.
8. Why is /usr/local/lib excluded when cross-compiling? Please add a comment (likewise for lib64)

Otherwise, it looks fine; I haven't been able to test it yet, though.
msg51365 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2007-06-03 23:20

"1. Specifying --build should not be necessary, as it should default to"

correct.

"2. what are the implications of AC_CHECK_TOOLS wrt. the current
AC_CHECK_PROGS invocations?"

AC_CHECK_TOOLS checks for TARGET-PROG, so it uses the proper cross
compile tools.

"There is a lot of logic to determine the C++ compiler...  "

Good point. This uses AC_CHECK_TOOLS too now.

"3. should we include config.sub? Can we share it easily with the libffi
one? Where do I get the most recent version of it?"

config.sub comes from autoconf, and if libffi uses it, it should be
the same one.

It's a bad idea to put generated files in SVN, as is done now; the
same holds for the toplevel configure.  Standard practice is to
include an autogen.sh script that will invoke the correct
autoconf/automake/aclocal/acheader/etc. voodoo to generate the
scripts.

"4. Where does the mapping of system names from -dumpmachine come from?
What would need to be done to eliminate this altogether? What about
ac_sys_release?"

The mapping has to be maintained by someone.  It's not very elegant,
but then again, there are some arbitrary mappings in configure.in in
the native build for ac_sys_release. IMO it would be better to do away
with this completely and use the autoconf/config.sub naming
exclusively.

"5. Isn't there some autoconf way for detecting a C compiler for the build
system? It shouldn't default to 'cc'."

I'm not aware of any; what do you think the default should be, if not
cc?  (There is AC_PROG_CC, but that will look for the X-compiler).

"6. I don't think the "skipping import check" warning is needed. Just
silently don't perform this check."

OK. - note that there are several other "skipping import check"
messages, for cygwin and Carbon.

"7. What is the meaning of CROSS_TARGET? In some place, it is used like
sys.platform (so it should take one of the possible values for
sys.platform), in configure.in, it is set to ac_sys_system.  I think you
should just use MACHDEP here."

Good point. I changed this.

"8. Why is /usr/local/lib excluded when cross-compiling? Please add a
comment (likewise for lib64)"

/usr/local/lib/ in general contains libraries for the build system,
not the target system. Linking them in will either generate bogus
linker errors ("wrong architecture binary") or worse create a module
which isn't loadable on the target platform, because the library is
missing there.

I've added a more generic mechanism, which filters include_dirs so
they can only start with $CROSS_ROOT.  Something similar should really
be applied to the linker directories, but I'm not sure which variable
to modify.

"Otherwise, it looks fine; I haven't been able to test it yet, though."

I've added some more information.  If you want to get a cross-compile
environtment, you could do

  mkdir gub ; cd gub ; git init ; git pull git://git.sv.gnu.org/lilypond.git gub:
  make -f lilypond.make PLATFORMS="freebsd-x86" cross-compilers

this will leave a cross compiler in
gub/target/freebsd-x86/usr/cross/bin

File Added: cross-55748.patch
msg56846 - (view) Author: Scott Tsai (scott.tsai) Date: 2007-10-27 06:55
cross-2.5.1.patch
forward ported cross-55748.patch to Python-2.5.1
Note that building on x86_64 for mipsel-linux still fails with:
gcc -c -I. -I./Include  -o Parser/acceler.x Parser/acceler.c
In file included from ./Include/Python.h:57,
                 from ./Include/pgenheaders.h:10,
                 from Parser/acceler.c:13:
./Include/pyport.h:734:2: error: #error "LONG_BIT definition appears
wrong for platform (bad gcc/glibc config?)."
make: *** [Parser/acceler.x] Error 1

The tools meant to run on the build machine can't be using the same
pyconfig.h as the onces that are meant to run on the host machine.
msg56847 - (view) Author: Scott Tsai (scott.tsai) Date: 2007-10-27 11:07
I messed up while generating cross-2.5.1.patch last time.
Added a hackish way to set "disabled_module_list" in setup.py from
corresponding environment variable.
msg56848 - (view) Author: Scott Tsai (scott.tsai) Date: 2007-10-27 11:11
Grumble, uploaded wrong version of patch.
msg58340 - (view) Author: John Stowers (nzjrs) Date: 2007-12-10 04:23
Hello, 

I recently tried this in combination with jhbuild, cross compiling with
mingw (to built some python gtk extensions). I tried you patch against
the 2.5.1 version and recieved the following error:

checking for /dev/ptmx... yes
checking for /dev/ptc... no
checking for %zd printf() format support... configure: error: cannot run
test program while cross compiling
See `config.log' for more details.
*** error during stage configure of python25: ########## Error running
./configure --prefix /home/john/jhbuild/win32/build/  
--build=i686-pc-linux-gnuaout --host=i586-pc-mingw32msvc
--target=i586-pc-mingw32msvc --disable-docs --enable-all-warnings 
AR="/usr/bin/i586-mingw32msvc-ar" 
RANLIB="/usr/bin/i586-mingw32msvc-ranlib" 
STRIP="/usr/bin/i586-mingw32msvc-strip" 
AS="/usr/bin/i586-mingw32msvc-as" 
DLLTOOL="/usr/bin/i586-mingw32msvc-dlltool" 
OBJDUMP="/usr/bin/i586-mingw32msvc-objdump" 
NM="/usr/bin/i586-mingw32msvc-nm" 
WINDRES="/usr/bin/i586-mingw32msvc-windres"  *** [1/1]

I noticed that the other reference for cross compiling pthon 2.5 at
http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/
patched this check out
msg58341 - (view) Author: John Stowers (nzjrs) Date: 2007-12-10 05:17
Sorry, I should have clarified further in my last comment. Looking over
the configure script I don't recognize the %zd test as one that could be
circumvented by supplying a config.cache file with the appropriate values.

How do I escape this limitation?
msg58342 - (view) Author: Scott Tsai (scott.tsai) Date: 2007-12-10 05:58
John,
set "ac_cv_printf_zd_format".
In general, read the configure.in source.

On Dec 10, 2007 1:17 PM, John Stowers <report@bugs.python.org> wrote:
>
> John Stowers added the comment:
>
> Sorry, I should have clarified further in my last comment. Looking over
> the configure script I don't recognize the %zd test as one that could be
> circumvented by supplying a config.cache file with the appropriate values.
>
> How do I escape this limitation?
>
>
> _____________________________________
> Tracker <report@bugs.python.org>
> <http://bugs.python.org/issue1597850>
> _____________________________________
>
msg58414 - (view) Author: John Stowers (nzjrs) Date: 2007-12-11 04:14
I created config.cache and populated it with 
ac_cv_printf_zd_format=yes

before executing the following

./configure --prefix /home/john/jhbuild/win32/build/  
--build=i686-pc-linux-gnuaout --host=i586-pc-mingw32msvc
--target=i586-pc-mingw32msvc --disable-docs --enable-all-warnings 
AR="/usr/bin/i586-mingw32msvc-ar" 
RANLIB="/usr/bin/i586-mingw32msvc-ranlib" 
STRIP="/usr/bin/i586-mingw32msvc-strip" 
AS="/usr/bin/i586-mingw32msvc-as" 
DLLTOOL="/usr/bin/i586-mingw32msvc-dlltool" 
OBJDUMP="/usr/bin/i586-mingw32msvc-objdump" 
NM="/usr/bin/i586-mingw32msvc-nm" 
WINDRES="/usr/bin/i586-mingw32msvc-windres" --config-cache

ac_cv_printf_zd_format=yes ./configure ..... doesn't work either

Is it necessary to rm configure && autoconf ? Im afraid I cant get this
to work...
msg61334 - (view) Author: Christopher Friedt (cfriedt) Date: 2008-01-20 19:08
I can confirm what John Stowers experienced with ac_cv_printf_zd

Did someone forget to run autoconf afterward?

When I did, retrying configure again returned an error saying that
config.sub was missing.

I made configure just write out a yes to the ac_cv_printf_zd_format. 

After running 'make', the build failed saying 

make: *** No rule to make target `Parser/acceler.@O_FOR_BUILD">Parser/acceler.@O_FOR_BUILD@', needed
by `Parser/pgen@EXEEXT_FOR_BUILD">Parser/pgen@EXEEXT_FOR_BUILD@'.  Stop.
msg73095 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-09-12 15:49
> In particular, I think that X-compiling is a common request

added another one to the list.

justification: pywebkitgtk cross-compiling for win32, using mingw32.
i'm not paying for microsoft license fees, i'm not paying for a
computer capable of running msvc, i'm not paying for the bandwidth
to download msvc and/or set it up.

much happier with mingw32, but mingw32 runs like an absolute dog
under wine - and often wine_server hangs.  (at least a 20x performance
hit for running msys and mingw32 under wine).

so... that leaves cross-compiling.

i need the development libraries from python 2.5 in order to create
a windows version of pywebkitgtk.
msg73111 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-09-12 18:23
the cross-compile fails on Parser/acceler.c
the reason is because the included file, pyconfig.h,
has "#define gid_t int" for use by the mingw32 compiler,
which is... bad!
removing gid_t from pyconfig.h bizarrely fixes the
compile without.. so far.. any issues....
msg73112 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-09-12 18:27
pyport.h line 773 - commenting out the test for LONG_BIT != 8 *
SIZEOF_LONG - we're cross-compiling amd64 host, target mingw32 - 32-bit.
msg73113 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-09-12 18:34
line 199 of thread_pthread.h  and line 221:
Python/thread_pthread.h:200: error: aggregate value used where an
integer was expected

hmmm... maybe this is due to me using mingw32 based on gcc 4.4.4.

well, a quick #if 0 seems to fix it :)
msg73114 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-09-12 18:35
posixmodule.c - line 2328:
add this:

#if ( defined(__MINGW32__) || defined(__WATCOMC__) ||
defined(PYCC_VACPP) ) && !defined(__QNX__)
    res = mkdir(path);
#else
    res = mkdir(path, mode);
#endif
msg73115 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-09-12 18:38
posixmodule: line 3530:

#ifdef __MINGW32__
    master_fd = open(DEV_PTY_FILE, O_RDWR); /* open master */
#else
    master_fd = open(DEV_PTY_FILE, O_RDWR | O_NOCTTY); /* open master */
#endif

not sure i should be compiling posixmodule under mingw32 :)
msg73116 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-09-12 18:40
line 6193:

#if !defined(__MINGW32__) && !defined(MS_WINDOWS) && defined(HAVE_FCNTL_H)
msg74372 - (view) Author: rwmjones (rwmjones) Date: 2008-10-06 13:56
I'm trying to use this patch to cross-compile Python under
Fedora, using the Fedora-MinGW project
(http://fedoraproject.org/wiki/MinGW)

IMHO the documentation is confusing.  It sounds from the
docs like CC_FOR_BUILD should be the native system
compiler -- eg. all the examples show CC_FOR_BUILD=gcc

But I didn't get very far at all doing that.  It seems that
what you mean is CC_FOR_BUILD=i686-pc-mingw32-gcc (ie. the
cross-compiler).

Also the extra patches added by lkcl are necessary to get
posixmodules.c to compile.  I'm trying 2.5.2 at the moment,
and while I haven't succeeded getting it all compiled yet,
it's very definitely necessary to fix posixmodules.c
msg74374 - (view) Author: Luke Kenneth Casson Leighton (lkcl) Date: 2008-10-06 14:52
ok.

what's not explained, and isn't clear, is exactly whether you're
supposed to - or even _capable_ of - cross-compiling the _entire_
python sourcecode base with mingw32, or whether you're supposed to
get _just_ the python.exe interpreter, and, maybe if you're lucky,
libpython.a (or libpython.dll - whatever).

i got quite a long way, as you can see, in cross-compiling
posixmodule.c _by mistake_ - before realising that the whole
python sourcecode base is completely inadequately set up for
cross-compiling, but is specialised "hard-coded" to compile
python under msvc _only_.

so, when doing the cross-compile, it was actually being detected
as a _unix_ compile, not a _win32_ compile.

#define WIN32 wasn't even being listened to.

the end result was that entire areas of posixmodule.c were
being compiled when they shouldn't have been.

later, i tried to correctly #define WIN32 (or whatever was
required), only to find that the mingw32 libraries don't
support many of the necessary functions.  for example, it's
assumed that crypto libraries exist, and many other things.

all in all, it didn't go too well :)
msg74375 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-06 15:08
So what's the status of this? Who is still interested in seeing what
patches considered?
msg74377 - (view) Author: rwmjones (rwmjones) Date: 2008-10-06 15:15
I'm very slowly working up a patch here (again 2.5.2).

Since I haven't actually got even python.exe compiled
yet I can't promise anything, but you never know ...
msg74457 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2008-10-07 15:00
I'm still interested in this, but the last time I did anything, I jumped
through all the hoops (see conversation here), and not a single change
was put into trunk. I'm not very enthousiastic about spending a lot time
on this again.
msg74458 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2008-10-07 15:03
@Luke

the compiling strategy for Python (IIRC) is to compile everything,
including modules that will never work, and use compiler errors as a
signal to not include a module in the result. 

this is what I end up with for 2.4

./usr/bin/libpython2.4.dll
./usr/bin/imageop.dll
./usr/bin/_codecs_hk.dll
./usr/bin/_codecs_jp.dll
./usr/bin/_heapq.dll
./usr/bin/_random.dll
./usr/bin/cPickle.dll
./usr/bin/cStringIO.dll
./usr/bin/regex.dll
./usr/bin/collections.dll
./usr/bin/_locale.dll
./usr/bin/_testcapi.dll
./usr/bin/_codecs_tw.dll
./usr/bin/pyexpat.dll
./usr/bin/_hotshot.dll
./usr/bin/mmap.dll
./usr/bin/math.dll
./usr/bin/binascii.dll
./usr/bin/array.dll
./usr/bin/smtpd.py
./usr/bin/cmath.dll
./usr/bin/audioop.dll
./usr/bin/_codecs_kr.dll
./usr/bin/parser.dll
./usr/bin/itertools.dll
./usr/bin/_csv.dll
msg74472 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-07 19:49
> the compiling strategy for Python (IIRC) is to compile everything,
> including modules that will never work, and use compiler errors as a
> signal to not include a module in the result. 

I don't think this can work in the cross-compilation case, though
(or the entire setup.py part of it can't really work). It uses the
target's python binary to run setup.py, compiles the modules, and then
tries to load them. In a cross-compilation case, you can't run the
target python binary, and even if you use a host python instead, you
can't load the target extension modules (unless the "cross" compilation
is for the same microprocessor and operating system - a case I'm not
very much interested in).
msg74475 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-10-07 19:56
I found the patch cross-2.5.1.patch as too limited.
I'm interesting in this topic and I post patch in issue3871 that
continue work from issue841454 and issue1412448.
msg74476 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-10-07 20:06
Martin meaning of target and host is different.
There is no reason to use "Canadian Cross": build->host->target.
It is about more simple cross-compilation case: build->host.

About loading of modules in build environment: some OS-es can run
binaries from other OS-es.
msg74480 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-07 20:23
> Martin meaning of target and host is different.
> There is no reason to use "Canadian Cross": build->host->target.
> It is about more simple cross-compilation case: build->host.

Terminology issues aside, I hope people still will understand my
objection. I meant it this way:
- host system: system hosting Python build process
- target system: system intended to run resulting Python executable
Whether that conforms to autoconf should be irrelevant, as long
as we aren't talking about configure.in changes.

> About loading of modules in build environment: some OS-es can run
> binaries from other OS-es.

Sure. However, I don't think this is the general case for
cross-compilation, and I don't think cross-compilation support in
Python should assume this is possible. Instead, cross-compilation
should assume that build system and target system have anything
in common (except that target headers and for-target cross tools
are installed on the build system).
msg74483 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-10-07 20:43
Now in mingw case the common is "python posix build system".
If the cross-compilation work what is problem to build in native
environment? Personally I prefer to build in cross environment. It is
convenient.

There is no problem to run python tests in the native environment.
In emulated environment most of the python test run smoothly.
msg74520 - (view) Author: rwmjones (rwmjones) Date: 2008-10-08 08:32
Just to clarify, in the MinGW case we are interested in:

"build" = Fedora Linux, usually i386 or x86-64 (but not always)
"host" = Windows i386

We can, to a limited extent, run the host binaries on the
build system, using Wine (the Windows emulator).  This doesn't
always work, and in any case is usually regarded as the wrong
thing to do because Wine is a very large and complicated build
dependency which requires, amongst other things, a working X
server.  Also since Wine doesn't run on anything which is
not x86-like, if we have to run Wine during the build then we
cannot cross-compile from other Fedora build systems, specifically
ppc and sparc.
msg74523 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-10-08 10:08
Hi rwmjones, 
Please, could you test patch from issue3871 - python modules are build
as setup.py is run from python found on the build system. So I don't
expect issue with ppc and sparc. Minor issue is pgen.exe - work around
touch grammar files.
msg76177 - (view) Author: Stephan Raue (sraue) Date: 2008-11-21 12:31
when i crosscompile Python 2.6 with Patch cross-2.6-0.7.diff which is 
based on cross-2.5.1.patch  i become follow error: 

ld -s -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-
prototypes -I. -IInclude -I./Include build/temp.linux-i686-
2.5/home/stephan/OpenELEC/build.i386.uClibc/Python-
2.6/Modules/_multiprocessing/multiprocessing.o build/temp.linux-i686-
2.5/home/stephan/OpenELEC/build.i386.uClibc/Python-
2.6/Modules/_multiprocessing/socket_connection.o build/temp.linux-i686-
2.5/home/stephan/OpenELEC/build.i386.uClibc/Python-
2.6/Modules/_multiprocessing/semaphore.o -L/usr/lib -lpython2.5 -o 
build/lib.linux-i686-2.5/_multiprocessing.so
ld: unrecognized option '-DNDEBUG'
ld: use the --help option for usage information
creating build/temp.linux-i686-2.5/libffi
checking build system type... i686-pc-linux-gnu
checking host system type... i686-pc-linux-gnu
checking target system type... i686-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... 
/home/stephan/OpenELEC/build.i386.uClibc/toolchain/bin/i686-geexbox-
linux-uclibc-gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... configure: error: cannot run C 
compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details.
Failed to configure _ctypes module

what can i do libffi compiles with --host and --build

when i compile libffi standalone and run configure with --with-system-
ffi the error is the same.
msg77151 - (view) Author: n03702 (n03702) Date: 2008-12-06 17:17
There is port of cross-2.6-0.7.diff patch for python 3.0 final
msg83052 - (view) Author: Joshua Kinard (kumba) Date: 2009-03-03 00:58
Anyone gotten farther on getting Python-2.5.x to cross-compile?  I'm
trying to get x86_64-pc-linux-gnu --> mipsel-unknown-linux-gnu, and
after some hacking at the last updated cross-2.5.1.patch, plus a fix for
the %zd printf bugaboo, plus adding in config.sub/config.guess files,
I'm able to get it moving a little.  But I'm running into the same
failure as described in Message #56846 and I'm not quite sure how to
properly work around that.

Can't "hack" around it -- the entire build is automated from a Gentoo
ebuild, so I need to be able to tweak something in Makefile.pre.in or
configure.in to fix this...somehow.

Ran some quick checks, and even with passing -I/usr/include to
CPPFLAGS_FOR_BUILD, and running the host compiler directly on the
command line, It's picking up wrong values for LONG_BIT and SIZEOF_LONG
(I think).

LONG_BIT = 64
SIZEOF_LONG = 4

So the test LONG_BIT != (SIZEOF_LONG * 8) succeeds and hits the #error
in pyport.h

Can't use a newer Python, including 2.6, 2.7-trunk, or even 3.0. 
Gentoo's primary package manager, portage, is currently dependent on
2.5.x (we're using 2.5.4 specifically right now).  So Roumen's patch
doesn't work at all (I've already tried backporting).

Ideas perhaps?
msg110394 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-15 22:41
I understand that cross-compilation is not supported so this must now be aimed at 3.2.
msg114558 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-21 20:13
Distutils is normally frozen for new features, but in this case the changes are small and useful enough to warrant an exception in my opinion (provided the patch is ported to 3.2 and gets a positive review). Tarek, do you agree?
msg114572 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-08-21 21:49
> the changes are small
which patch are you referring to? They look quite large to me.
msg114573 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-21 21:50
cross-3.0-0.7.diff only changes a few lines in build_ext. I was specifically talking only about distutils changes.
msg114576 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-08-21 22:20
For cross-3.0-0.7.diff, we would need a real name and a contributor agreement.

Of course, attribution is muddy here; this literally goes back to sraue's patch, which in turn goes back to scott.tsai's patch.

I'm still uncertain what it is that this patch actually achieves.
msg114752 - (view) Author: Roumen Petrov (rpetrov) * Date: 2010-08-23 22:39
-1 for the patch (after review of cross-3.0-0.7.diff) :
1) AC_CHECK_TOOLS(CC,gcc cc) and AC_CHECK_TOOLS(CXX,g++ c++) is bogus 
2) "$CC -dumpmachine" when is added AC_CANONICAL_HOST is bogus
3) if (strcmp(buffe,me) "123")) is buggy

Good point is setup.py is to exclude default searches for header/libraries in /usr/XXX directories . Note that this is requested in other issues not related to cross compilation.

About pgen build i'm not sure that is wort to cross-build as there is no reason to recreate files , right ? To me this is only dependency issue.
msg181186 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-02-02 17:46
Proposed patch is mostly for cross compilation in general. Now this is implemented differently and I think that all proposed updates are already addressed.

Also I can not  see relation with gcc( mingw ) builds.

What about to close issue as fixed ?
msg181209 - (view) Author: Han-Wen Nienhuys (hanwen) * Date: 2013-02-02 20:00
yeah, whatever.

(only 7 years to close an issue. Yay for open-source.)
History
Date User Action Args
2013-02-02 20:00:40hanwensetstatus: open -> closed

messages: + msg181209
2013-02-02 17:46:28rpetrovsetmessages: + msg181186
2011-05-20 14:24:38gregory.p.smithsetnosy: - gregory.p.smith
2011-05-20 10:08:16wrobellsetnosy: + wrobell
2011-03-25 14:57:49eric.araujosetversions: + Python 3.3, - 3rd party, Python 3.2
2011-02-13 16:00:23alexissetnosy: + alexis
2010-09-30 00:26:18eric.araujosetversions: + 3rd party
2010-08-23 22:39:20rpetrovsetmessages: + msg114752
2010-08-21 22:20:58loewissetmessages: + msg114576
2010-08-21 21:50:59eric.araujosetmessages: + msg114573
2010-08-21 21:49:06amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg114572
2010-08-21 20:13:55eric.araujosetassignee: tarek
type: enhancement
components: + Distutils, Distutils2

nosy: + tarek, eric.araujo
messages: + msg114558
stage: patch review
2010-07-15 22:41:11BreamoreBoysetnosy: + BreamoreBoy

messages: + msg110394
versions: + Python 3.2, - Python 2.6
2009-07-31 21:13:38gregory.p.smithsetnosy: + gregory.p.smith
2009-03-03 00:58:49kumbasetnosy: + kumba
messages: + msg83052
2008-12-06 17:17:24n03702setfiles: + cross-3.0-0.7.diff
nosy: + n03702
messages: + msg77151
2008-11-21 12:32:25srauesetfiles: + cross-2.6-0.7.diff
nosy: + sraue
messages: + msg76177
versions: + Python 2.6, - Python 2.5
2008-10-08 10:08:45rpetrovsetmessages: + msg74523
2008-10-08 08:32:07rwmjonessetmessages: + msg74520
2008-10-07 20:43:48rpetrovsetmessages: + msg74483
2008-10-07 20:23:10loewissetmessages: + msg74480
2008-10-07 20:06:31rpetrovsetmessages: + msg74476
2008-10-07 19:56:27rpetrovsetnosy: + rpetrov
messages: + msg74475
2008-10-07 19:49:26loewissetmessages: + msg74472
2008-10-07 15:03:52hanwensetmessages: + msg74458
2008-10-07 15:00:55hanwensetmessages: + msg74457
2008-10-06 15:15:31rwmjonessetmessages: + msg74377
2008-10-06 15:08:42loewissetmessages: + msg74375
2008-10-06 14:52:27lkclsetmessages: + msg74374
2008-10-06 13:56:23rwmjonessetnosy: + rwmjones
messages: + msg74372
2008-09-12 18:40:23lkclsetmessages: + msg73116
2008-09-12 18:38:36lkclsetmessages: + msg73115
2008-09-12 18:35:49lkclsetmessages: + msg73114
2008-09-12 18:34:22lkclsetmessages: + msg73113
2008-09-12 18:27:14lkclsetmessages: + msg73112
2008-09-12 18:23:36lkclsetmessages: + msg73111
2008-09-12 15:49:55lkclsetnosy: + lkcl
messages: + msg73095
2008-01-20 19:08:26cfriedtsetnosy: + cfriedt
messages: + msg61334
2008-01-05 19:59:41christian.heimeslinkissue1339673 superseder
2007-12-11 04:14:01nzjrssetmessages: + msg58414
2007-12-10 05:58:53scott.tsaisetmessages: + msg58342
2007-12-10 05:17:51nzjrssetmessages: + msg58341
2007-12-10 04:23:19nzjrssetnosy: + nzjrs
messages: + msg58340
2007-10-27 11:11:01scott.tsaisetfiles: + cross-2.5.1.patch
messages: + msg56848
2007-10-27 11:07:30scott.tsaisetfiles: + cross-2.5.1.patch
messages: + msg56847
2007-10-27 06:55:56scott.tsaisetfiles: + cross-2.5.1.patch
nosy: + scott.tsai
messages: + msg56846
2006-11-16 16:57:12hanwencreate