classification
Title: distutils compiler not handling spaces in path to output/src files
Type: behavior Stage:
Components: Distutils Versions: Python 3.1, Python 3.0, Python 2.7, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: tarek Nosy List: Thorney, ceball, loewis, tarek (4)
Priority: normal Keywords patch

Created on 2008-12-03 21:58 by Thorney, last changed 2009-03-18 13:57 by ceball.

Files
File name Uploaded Description Edit Remove
distutils_compiler_quoting.patch Thorney, 2008-12-03 21:58 Patch to quote src and object args in distutils c compiler
python_extension_bug.rar Thorney, 2009-01-12 01:20 Example use of distutils that fails with spaces in the source code filename.
Messages (6)
msg76848 - (view) Author: Brian Thorne (Thorney) Date: 2008-12-03 21:58
I found this bug when using scipy's weave and having some inline C/C++
code. Weave uses distutils to compile the source in its own directory,
python was installed under "program files". Unfortunately it doesn't
seem to handle spaces in the path to either the output or the temporary
source files.
I checked that this was the only problem by copying the weave directory
from site-packages and put it in another directory without spaces in the
path - weave and distutils then worked fine.
I then tried the g++ command seen in the stack trace at the command line
and got the same error - so altered the command by correctly quoting the
 -c "path/to source/a.c" and the -o objects.
This worked so set out changing it in distutils.

The two files I altered were unixcompiler.py (surprising as I am under
windows!) and cygwinccompiler.py 

This was clearly an older stable build so I checked out the python trunk
r67510 on windows (32bit), made a patch, and tested with my weave
dependant code.
(I am not familiar enough with distutils to make a standalone test).

The same method to cause a problem has the same effect in ubuntu - the
build directory seems to default to users home, but as the source path
has a space in it g++ gets confused.
msg77510 - (view) Author: Martin v. Löwis (loewis) Date: 2008-12-10 08:54
The patch hasn't been accepted for the trunk, yet, so it is not a
candidate for a 2.5 backport.
msg78632 - (view) Author: Tarek Ziadé (tarek) Date: 2008-12-31 16:46
Hi. I am not familiar with weave. Could you provide a small sample of
code that raises this issue. This way, I will be able to write the
standalone test we can integrate in distutils together with your fix.
msg79641 - (view) Author: Brian Thorne (Thorney) Date: 2009-01-12 01:20
Ok - this bug has nothing to do with weave itself. I think any extension
written in C or C++ that has a space in the filename will raise the
error in both windows and gnu/linux.
Here is a simple c extension with and without spaces. The one with
spaces in the filename fails to build on standard python. Whereas the
one without should build fine. Did for me on windows 2000 on both python
2.5 and 2.6 with and without spaces in their paths.
msg80066 - (view) Author: Tarek Ziadé (tarek) Date: 2009-01-18 00:45
I couldn't reproduce the problem yet, using your sample.

I tried to compile on Mac OSX (Python 2.5 GCC) and Windows XP (Python
2.5 , MinGW 5.1.4 and SWIG 1.2.3) and it worked well.

The output was right, (the .o file has a space, but the pyd/so files
come out properly), and I could use the gcd() function without any problem.

The call to the compiler quotes the c file correctly:

    ...
    c:\Program Files\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall
-Ic:\Python25\include -Ic:\Python25\PC -c "ex ample.c" -o
"build\temp.win32-2.5\Release\ex ample.o"
    ...

This is done automatically with distutils.spawn._nt_quote_args under
Windows for instance. And this is basically what your patch is doing.

When does it crash exactly for you ? Do you have a traceback ?
msg83748 - (view) Author: C. E. Ball (ceball) Date: 2009-03-18 13:57
I also found this bug while using SciPy's weave on Windows XP to compile
some inline c/c++ code.

Thorney's distutils_compiler_quoting.patch fixed the problem for me.


Here is the relevant part of the traceback (I hope):
"""
Found executable C:\Program Files\Topographica\python_topo\mingw\bin\g++.exe
g++.exe:
c:\docume~1\ceball\locals~1\temp\ceball\python25_intermediate\compiler_
894ad5ed761bb51736c6d2b7872dc212\Release\program: No such file or directory
g++.exe:
files\topographica\python_topo\lib\site-packages\weave\scxx\weave_imp.o:
 No such file or directory
"""

Chris
History
Date User Action Args
2009-03-18 13:57:09ceballsetnosy: + ceball
messages: + msg83748
2009-01-18 00:45:37tareksetmessages: + msg80066
2009-01-12 01:20:39Thorneysetfiles: + python_extension_bug.rar
messages: + msg79641
2008-12-31 16:46:30tareksetpriority: normal
assignee: tarek
messages: + msg78632
nosy: + tarek
2008-12-10 08:54:57loewissetnosy: + loewis
messages: + msg77510
versions: - Python 2.5, Python 2.4, Python 2.3, Python 2.2.3, Python 2.2.2, Python 2.2.1, Python 2.2, Python 2.1.2, Python 2.1.1, Python 2.5.3
2008-12-03 21:58:56Thorneycreate