_sysconfigdata is generated in srcdir, not builddir, so if you do two consecutive differently builds in different builddirs and using the same srcdir, then the _sysconfigdata of the second build wins. _sysconfigdata.py has to be built in the builddir, not the srcdir.

looks like the module can be put into $builddir/`cat pybuilddir.txt`. However pybuilddir.txt is written/computed by the just built python. So either

 - implement distutils.util.get_platform in configure.ac

 - or write a temporary pybuilddir.txt, call the just built
   python -c 'print("build/lib.%s-%" % (distutils.util.get_platform(), sys.version[:3]))' to write the final pybuilddir.txt, move the _sysconfigdata.py to this location.
   and remove the pybuilddir.txt writing in setup.py

proposed patch, tested with configure, build and install

The patch is rather ugly, I think. You should arrange for sysconfigdata to be generated directly at the right place instead (not sure how, perhaps it should be done from setup.py).

Also, there's no need for this to be a release blocker.

(Once this issue is resolved, a permanent fix for the minor OS X test case failure of Issue15188 can be developed and applied.)

while I appreciate the fix for #13150, it's the patch for #13150 which introduces the ugliness to build the file in the srcdir in the first place.

re-setting the bug severity. the current behaviour can result in a bogus installation; please let the release manager decide on it.

like other platform dependent files _sysconfigdata.py should be installed in plat-*

> like other platform dependent files _sysconfigdata.py should be installed in plat-*

What are you talking about? plat-* files are only OS-specific, while
_sysconfigdata also depends on configure options and other stuff.

The bug certainly is a valid annoyance, but I won't hold up beta2 for it.

It seems there is no urge to fix this before 3.3 -- fine with me.

If sysconfig._generate_posix_vars() creates the build directory and pybuilddir.txt (instead of setup.py) then it could write _sysconfigdata.py in the correct place.  Then setup.py would not have to mess with its own sys.path.

One issue with having _sysconfigdata.py in the build dir is that distutils copies it (along with everything else in the build dir) to .../lib/python3.x/lib-dynload.  So I think Matthias' patch results in two copies of _sysconfigdata.py being installed (plus __pycache__/_sysconfigdata.pyc).

Alternative patch.

I did not test the patch but it does not look bad.

looks ok, two issues are:

builddir should have added
  +("-pydebug" if hasattr(sys, "gettotalrefcount") else "")

the installation installs this to the lib-dynload directory, which maybe is not desired.

> builddir should have added
>  +("-pydebug" if hasattr(sys, "gettotalrefcount") else "")

Oops, yes.

> the installation installs this to the lib-dynload directory, which 
> maybe is not desired.

The lines

+	-rm $(DESTDIR)$(DESTSHARED)/_sysconfigdata.py
+	-rm -r $(DESTDIR)$(DESTSHARED)/__pycache__

in the patch remove them from lib-dynload.

Updated patch.

Try again.

looks fine. from my point of view this should go to the 3.3 branch after the 3.3.0 release, and to the trunk.

The previous 'alt_sysconfigdata.patch' didn't apply cleanly to 3.3/3.x.  I've attached an updated patch that does.

Any objections to applying this to 3.3 -> 3.x now that the freeze is over?

New changeset 24d52d3060e8 by Trent Nelson in branch '3.3':
Issue #15298: ensure _sysconfigdata is generated in build directory,
http://hg.python.org/cpython/rev/24d52d3060e8

New changeset f85c3f4d9b98 by Trent Nelson in branch 'default':
Merge issue #15298: ensure _sysconfigdata is generated in build directory,
http://hg.python.org/cpython/rev/f85c3f4d9b98

There seems to be a bootstrap issue here.  Building with ./configure --with-pydebug --prefix=... on OS X in a clean source directory (hg purge --all), 'make' makes it to building the static libpython .a but then dies on the sysconfig generate-posix-vars step:

ar rc libpython3.4dm.a Modules/_threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/_functoolsmodule.o  Modules/operator.o  Modules/_collectionsmodule.o  Modules/itertoolsmodule.o  Modules/_localemodule.o  Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/zipimport.o  Modules/faulthandler.o  Modules/symtablemodule.o  Modules/xxsubtype.o
/usr/bin/ranlib: file: libpython3.4dm.a(dynamic_annotations.o) has no symbols
/usr/bin/ranlib: file: libpython3.4dm.a(pymath.o) has no symbols
ranlib libpython3.4dm.a
ranlib: file: libpython3.4dm.a(dynamic_annotations.o) has no symbols
ranlib: file: libpython3.4dm.a(pymath.o) has no symbols
/usr/bin/clang   -framework CoreFoundation -o python Modules/python.o libpython3.4dm.a -ldl  -framework CoreFoundation     
./python -E -S -m sysconfig --generate-posix-vars
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 683, in <module>
    _main()
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 671, in _main
    _generate_posix_vars()
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 374, in _generate_posix_vars
    pybuilddir = 'build/lib.%s-%s' % (get_platform(), sys.version[:3])
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 651, in get_platform
    get_config_vars(),
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 520, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/py/dev/default/b10.7_t10.7_x4.3_cclang_d/unix/source/Lib/sysconfig.py", line 393, in _init_posix
    from _sysconfigdata import build_time_vars
ImportError: No module named '_sysconfigdata'

Ah, this is because of this OS X-specific snippet in sysconfig.get_platform():

    elif osname[:6] == "darwin":
        import _osx_support
        osname, release, machine = _osx_support.get_platform_osx(
                                            get_config_vars(),
                                            osname, release, machine)

get_config_vars() results in _init_posix() eventually being called, which attempts to import _sysconfigdata.

Ugh.  Looking into a patch now.

New changeset 66959d419369 by Trent Nelson in branch '3.3':
Issue #15298: fix an OS X bootstrap issue with _sysconfigdata.py.
http://hg.python.org/cpython/rev/66959d419369

New changeset 429774e8d9f9 by Trent Nelson in branch 'default':
Merge issue #15298: fix an OS X bootstrap issue with _sysconfigdata.py.
http://hg.python.org/cpython/rev/429774e8d9f9

That last commit fixes in-tree builds, but broke out-of-tree builds.  I've refactored the patch (sysconfig.py.patch) to fix the issue, but it's getting progressively more hacky, so I wanted to see what other people think before proceeding.

(That being said, it works a *lot* better than the previous solution of writing _sysconfigdata.py to the root builddir, then moving it.  That didn't work at all for out-of-tree builds.  This solution slots _sysconfigdata directly into sys.modules, which _init_posix() is more than happy with.)

New changeset 1250498db562 by Trent Nelson in branch '3.3':
Issue #15298: refactor previous fix from 66959d419369.
http://hg.python.org/cpython/rev/1250498db562

New changeset a0614e041bc8 by Trent Nelson in branch 'default':
Merge issue #15298.
http://hg.python.org/cpython/rev/a0614e041bc8

Can this be closed?

On OS X, Trent's fixes solved the bootstrap issue and _sysconfigdata.py is now created in buildir.  Closing.