New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
_sysconfigdata is generated in srcdir, not builddir #59503
Comments
_sysconfigdata is generated in srcdir, not builddir, so if you do two consecutive differently builds in different builddirs and using the same srcdir, then the _sysconfigdata of the second build wins. _sysconfigdata.py has to be built in the builddir, not the srcdir. |
looks like the module can be put into $builddir/
|
proposed patch, tested with configure, build and install |
The patch is rather ugly, I think. You should arrange for sysconfigdata to be generated directly at the right place instead (not sure how, perhaps it should be done from setup.py). Also, there's no need for this to be a release blocker. |
(Once this issue is resolved, a permanent fix for the minor OS X test case failure of bpo-15188 can be developed and applied.) |
like other platform dependent files _sysconfigdata.py should be installed in plat-* |
What are you talking about? plat-* files are only OS-specific, while |
The bug certainly is a valid annoyance, but I won't hold up beta2 for it. |
It seems there is no urge to fix this before 3.3 -- fine with me. |
If sysconfig._generate_posix_vars() creates the build directory and pybuilddir.txt (instead of setup.py) then it could write _sysconfigdata.py in the correct place. Then setup.py would not have to mess with its own sys.path. One issue with having _sysconfigdata.py in the build dir is that distutils copies it (along with everything else in the build dir) to .../lib/python3.x/lib-dynload. So I think Matthias' patch results in two copies of _sysconfigdata.py being installed (plus __pycache__/_sysconfigdata.pyc). |
Alternative patch. |
I did not test the patch but it does not look bad. |
looks ok, two issues are: builddir should have added the installation installs this to the lib-dynload directory, which maybe is not desired. |
Oops, yes.
The lines + -rm in the patch remove them from lib-dynload. |
Updated patch. |
Try again. |
looks fine. from my point of view this should go to the 3.3 branch after the 3.3.0 release, and to the trunk. |
The previous 'alt_sysconfigdata.patch' didn't apply cleanly to 3.3/3.x. I've attached an updated patch that does. Any objections to applying this to 3.3 -> 3.x now that the freeze is over? |
New changeset 24d52d3060e8 by Trent Nelson in branch '3.3': New changeset f85c3f4d9b98 by Trent Nelson in branch 'default': |
There seems to be a bootstrap issue here. Building with ./configure --with-pydebug --prefix=... on OS X in a clean source directory (hg purge --all), 'make' makes it to building the static libpython .a but then dies on the sysconfig generate-posix-vars step: ar rc libpython3.4dm.a Modules/_threadmodule.o Modules/signalmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/operator.o Modules/_collectionsmodule.o Modules/itertoolsmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/zipimport.o Modules/faulthandler.o Modules/symtablemodule.o Modules/xxsubtype.o
/usr/bin/ranlib: file: libpython3.4dm.a(dynamic_annotations.o) has no symbols
/usr/bin/ranlib: file: libpython3.4dm.a(pymath.o) has no symbols
ranlib libpython3.4dm.a
ranlib: file: libpython3.4dm.a(dynamic_annotations.o) has no symbols
ranlib: file: libpython3.4dm.a(pymath.o) has no symbols
/usr/bin/clang -framework CoreFoundation -o python Modules/python.o libpython3.4dm.a -ldl -framework CoreFoundation
./python -E -S -m sysconfig --generate-posix-vars
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/runpy.py", line 160, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/runpy.py", line 73, in _run_code
exec(code, run_globals)
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 683, in <module>
_main()
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 671, in _main
_generate_posix_vars()
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 374, in _generate_posix_vars
pybuilddir = 'build/lib.%s-%s' % (get_platform(), sys.version[:3])
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 651, in get_platform
get_config_vars(),
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 520, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 393, in _init_posix
from _sysconfigdata import build_time_vars
ImportError: No module named '_sysconfigdata' |
Ah, this is because of this OS X-specific snippet in sysconfig.get_platform():
get_config_vars() results in _init_posix() eventually being called, which attempts to import _sysconfigdata. Ugh. Looking into a patch now. |
New changeset 66959d419369 by Trent Nelson in branch '3.3': New changeset 429774e8d9f9 by Trent Nelson in branch 'default': |
That last commit fixes in-tree builds, but broke out-of-tree builds. I've refactored the patch (sysconfig.py.patch) to fix the issue, but it's getting progressively more hacky, so I wanted to see what other people think before proceeding. (That being said, it works a *lot* better than the previous solution of writing _sysconfigdata.py to the root builddir, then moving it. That didn't work at all for out-of-tree builds. This solution slots _sysconfigdata directly into sys.modules, which _init_posix() is more than happy with.) |
New changeset 1250498db562 by Trent Nelson in branch '3.3': New changeset a0614e041bc8 by Trent Nelson in branch 'default': |
Can this be closed? |
On OS X, Trent's fixes solved the bootstrap issue and _sysconfigdata.py is now created in buildir. Closing. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: