msg87494 - (view) |
Author: kxroberto (kxroberto) |
Date: 2009-05-09 13:29 |
with --inplace etc. build_ext.get_outputs returns wrong extension paths;
tries to computes them out of the air *again* and ..
Tools which need to further know the produced module file, e.g.
pyximport (Cython) do crash then ...
patch see attachment: stores and uses the real path in use.
writes a new attribute ext_filename to Extension object - should it be
underscored _ext_filename or ... ?
|
msg87495 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2009-05-09 13:36 |
can you provide a test case that demonstrates the problem ?
|
msg87506 - (view) |
Author: kxroberto (kxroberto) |
Date: 2009-05-09 19:15 |
>>> test_build_ext: The test must be run in a python build dir
don't have a build setup currently. maybe in future.
One can yet see in build_ext.py:
after "if self.inplace:" (line485)
there are 2 different computations for ext_filename
while computation in get_outputs() is not aware of self.inplace (line447
|
msg87524 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2009-05-10 12:26 |
I was talking about your use case (pyximport (Cython) do crash then ?)
If I can have a real example, with the output and the expected one,
I will build a test with it in Distutils before I fix it
|
msg87525 - (view) |
Author: kxroberto (kxroberto) |
Date: 2009-05-10 14:31 |
its with any .pyx (Cython) module , when after pyximport.install() and
injection of --inplace option a .pyx module is imported.
the relevant lines in pyxbuild.py are
dist.run_commands()
return dist.get_command_obj("build_ext").get_outputs()[0]
which use the buggy "get_outputs" and shall return the full path of
built module to pyximport.py :
so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod,
build_in_temp=build_in_temp,
pyxbuild_dir=pyxbuild_dir,
setup_args=sargs )
assert os.path.exists(so_path), "Cannot find: %s" % so_path
=> crash with "Cannot find..." before pyximport.load_module goes to
import it.
-
A stripped down test case should perhaps 'build_ext' any arbitrary
extension module with --inplace ( a option of base command 'build' ) and
something like ...
dist.get_command_obj("build_ext").inplace=1
dist.run_commands()
so_path = dist.get_command_obj("build_ext").get_outputs()[0]
assert os.path.isfile(so_path) and os.path.dirname(so_path) in ('','.')
... will produce a invalid so_path: not pointing to actual locally
in-place built xy.pyd/.so, but to a non-existing or old file in the
build folders
|
msg87643 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2009-05-12 17:17 |
done in r72585 and propagated in 2.6, 3.0, 3.1 branches
Thank you !
|
msg87654 - (view) |
Author: Daniel Diniz (ajaksu2) * |
Date: 2009-05-12 18:46 |
Tarek,
It looks like all buildbots are red because setup.py doesn't work with
this change:
running build
running build_ext
Traceback (most recent call last):
File "./setup.py", line 1896, in <module>
main()
File "./setup.py", line 1891, in main
'Lib/smtpd.py']
File
"/home2/buildbot/slave/trunk.loewis-sun/build/Lib/distutils/core.py",
line 149, in setup
dist.run_commands()
File
"/home2/buildbot/slave/trunk.loewis-sun/build/Lib/distutils/dist.py",
line 974, in run_commands
self.run_command(cmd)
File
"/home2/buildbot/slave/trunk.loewis-sun/build/Lib/distutils/dist.py",
line 994, in run_command
cmd_obj.run()
File
"/home2/buildbot/slave/trunk.loewis-sun/build/Lib/distutils/command/build.py",
line 132, in run
self.run_command(cmd_name)
File
"/home2/buildbot/slave/trunk.loewis-sun/build/Lib/distutils/cmd.py",
line 326, in run_command
self.distribution.run_command(command)
File
"/home2/buildbot/slave/trunk.loewis-sun/build/Lib/distutils/dist.py",
line 994, in run_command
cmd_obj.run()
File
"/home2/buildbot/slave/trunk.loewis-sun/build/Lib/distutils/command/build_ext.py",
line 348, in run
self.build_extensions()
File "./setup.py", line 103, in build_extensions
missing = self.detect_modules()
File "./setup.py", line 302, in detect_modules
add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
AttributeError: 'NoneType' object has no attribute 'library_dirs'
[31295 refs]
*** Error code 1
The following command caused the error:
case $MAKEFLAGS in \
*s*) CC='gcc' LDSHARED='gcc -shared' OPT='-g -Wall -Wstrict-prototypes'
./python -E ./setup.py -q build;; \
*) CC='gcc' LDSHARED='gcc -shared' OPT='-g -Wall -Wstrict-prototypes'
./python -E ./setup.py build;; \
esac
make: Fatal error: Command failed for target `sharedmods'
program finished with exit code 1
|
msg87662 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2009-05-12 22:09 |
Sorry about that, I was travelling and didn't see that the buildbots
turned red. (I ran the test but didn't build python again)
I have changed that because build_ext.compiler is a string option, e.g.
the compiler type. Turning it into a compiler instance when the command
is run twice it breaks because it is used as a string argument to
create a compiler.
Maybe a solution would be to rename the option name into "compiler_type"
and keep build_ext.compiler as a compiler instance.
|
msg87663 - (view) |
Author: Daniel Diniz (ajaksu2) * |
Date: 2009-05-12 23:12 |
Tarek,
Sorry about breaking the compiler option, I was aware that passing None
there would have that risk.
I think anything that breaks people's setup.py shouldn't go into
release26-maint without a really important motivation.
Is there a way to keep self.compiler as an attribute and make
new_compiler work correctly with it?
|
msg87669 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2009-05-13 07:55 |
new_compiler takes "compiler" as a compiler name. It would make much
sense to make it accept a compiler instance.
now for people setup.py, I wasn't aware that people are using
build_ext.compiler as a compiler, besides Python itself.
IMHO we should rename the compiler option to "compiler_type" then
provide an API build_ext.get_compiler() then fix Python setup.py file.
Then this should not go into 2.6 but in 2.7, and we should put back the
option in new_compiler like it was (it's wrong but at least it works if
you run the command just once, which is the most common case)
|
msg87670 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2009-05-13 08:12 |
typo:
It would make much... -> It would not make much...
|
msg87759 - (view) |
Author: Roumen Petrov (rpetrov) * |
Date: 2009-05-14 20:11 |
Proposed patch may fix another windows failure as after recent commit
"unresolved external symbol initfoo" seems to me fixed.
|
msg87761 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2009-05-14 20:27 |
Roumen, the problem you mentioned was existing because of the
empty c file in test_get_outputs. I have fixed it yesterday by creating a
simple c file containing a minimal code:
void inifoo(void) {};
can you check it works for you ?
|
msg143918 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2011-09-12 16:21 |
The fix you committed uses an absolute path for the C sources, which is invalid: sources should contain relative paths only.
|
msg213606 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2014-03-15 01:06 |
This bug is quite bad, and was reported as far back as 2005 on distutils-sig. If one runs “python setup.py build_ext --inplace build”, then for example runs tests, and then “python setup.py install” runs build again, which runs build_py (.py files are already in the build directory, do nothing) then build_ext (.so files are not in the build directory: build them), so extensions modules are installed. But if one has “[build_ext] inplace=1” in the setup.cfg, then the .so files will not be recompiled and not installed.
It would be interesting to test if bdist_wheel is affected, as bdist commands use the install command under the hood.
|
msg386281 - (view) |
Author: Steve Dower (steve.dower) * |
Date: 2021-02-03 18:10 |
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.
If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
|
|
Date |
User |
Action |
Args |
2022-04-11 14:56:48 | admin | set | github: 50227 |
2021-02-03 18:10:47 | steve.dower | set | status: open -> closed
nosy:
+ steve.dower messages:
+ msg386281
resolution: out of date stage: patch review -> resolved |
2014-03-15 01:06:19 | eric.araujo | set | assignee: tarek -> eric.araujo messages:
+ msg213606 components:
- Distutils2 versions:
+ Python 3.4, - 3rd party, Python 3.2 |
2011-09-12 16:24:19 | eric.araujo | link | issue9322 dependencies |
2011-09-12 16:21:31 | eric.araujo | set | messages:
+ msg143918 versions:
- Python 3.1 |
2011-03-21 01:40:43 | eric.araujo | set | nosy:
+ eric.araujo, alexis title: distutils build_ext.get_outputs returns wrong result (patch) -> distutils build_ext.get_outputs returns wrong result versions:
+ 3rd party, Python 3.2, Python 3.3, - Python 2.6, Python 3.0 components:
+ Distutils2 type: crash -> behavior stage: patch review |
2009-05-14 20:27:20 | tarek | set | messages:
+ msg87761 |
2009-05-14 20:11:26 | rpetrov | set | files:
+ issue5977-w32.patch nosy:
+ rpetrov messages:
+ msg87759
|
2009-05-13 08:12:46 | tarek | set | messages:
+ msg87670 |
2009-05-13 07:55:54 | tarek | set | messages:
+ msg87669 |
2009-05-12 23:12:51 | ajaksu2 | set | priority: critical -> normal
messages:
+ msg87663 |
2009-05-12 22:09:18 | tarek | set | nosy:
+ benjamin.peterson messages:
+ msg87662
|
2009-05-12 18:46:08 | ajaksu2 | set | status: closed -> open files:
+ build_ext._compiler.diff
nosy:
+ ajaksu2 messages:
+ msg87654
priority: critical |
2009-05-12 17:17:20 | tarek | set | status: open -> closed
messages:
+ msg87643 versions:
+ Python 2.6, Python 3.0 |
2009-05-10 14:31:31 | kxroberto | set | messages:
+ msg87525 |
2009-05-10 12:26:47 | tarek | set | messages:
+ msg87524 |
2009-05-09 19:15:23 | kxroberto | set | messages:
+ msg87506 |
2009-05-09 13:36:39 | tarek | set | messages:
+ msg87495 versions:
+ Python 3.1, Python 2.7, - Python 2.6 |
2009-05-09 13:29:48 | kxroberto | create | |