When testing config options, it is often useful to build python in a directory different from the root of the source tree. This is supported by autoconf based builds as follows: you cd to the desired directory run configure with an explicit path. For example:

$ cd ../altdir
$ ../python-source/configure
$ make

However, this fails for python 3.1 and 3.2 with the following error:

i686-apple-darwin10-gcc-4.2.1: Parser/tokenizer_pgen.o: No such file or directory
i686-apple-darwin10-gcc-4.2.1: Parser/printgrammar.o: No such file or directory
i686-apple-darwin10-gcc-4.2.1: Parser/pgenmain.o: No such file or directory
make: *** [Parser/pgen] Error 1

I think the problem is with the following explicit rules in Makefile:

Parser/tokenizer_pgen.o: $(srcdir)/Parser/tokenizer.c

Parser/pgenmain.o: $(srcdir)/Include/parsetok.h

It looks like these rules are not VPATH friendly. Per Antoine's suggestions, adding build experts to the nosy list.

It works here (Linux). Did you try make distclean first? Also, try to remove the various files created by configure.

(what is VPATH?)

It turns out that make only gets confused when target files (Parser/pgenmain.o etc.) exist in the source directory. Thus an obvious workaround is to build all versions in alternative directories or delete object files from source tree before alternative builds.

This is still fragile and does not explain why for some targets, VPATH candidates are ignored and for some are not.

(what is VPATH?)

VPATH is a search path that make uses to find targets and prerequisites. configure sets it to its own location when invoked from a directory other than cwd.

Please could you remove object files from source tree then try again.

This is invalid issue.

On Wed, Sep 15, 2010 at 5:45 PM, Roumen Petrov <report@bugs.python.org> wrote:
..

Please could you remove object files from source tree then try again.

It looks like you missed my penultimate comment.

This is invalid issue.

At the very least, Building Python [1] section can be expanded to
cover building outside of the source tree. Some developers are not
even aware that this is supported.

[1] http://docs.python.org/using/unix.html#building-python

Does this apply to 2.7 too?

Does this apply to 2.7 too?

I think it does, but it would be hard to sell any improvements in this area as a bug fix.

FYIW, the OS X installer build script (Mac/BuildScript/build-installer.py for years has, and continues to, depend on building outside of the source directory. When I build one, I always start with a clean source directory. There are so many opportunities for the configure and make stages to go wrong that it seems not worth the effort to try to plug all of the holes of the type you have identified here when attempting to build with a "dirty" source directory. I wouldn't trust using one.

Here is a related bug:

$ make patchcheck
./python.exe ../py3k-commit/Tools/scripts/patchcheck.py
Getting the list of files that have been added/changed ... need a checkout to get modified files
[49399 refs]
make: *** [patchcheck] Error 1

I am working on a patch to make patchcheck use os.path.join(sysconfig.get_config_var('srcdir'), etc.) to look for the .hg dir and open files (to do its checks) with the right paths.

New changeset 313a71664781 by Éric Araujo in branch '3.2':
Let “make patchcheck” work for out-of-dir builds (bpo-9860)
http://hg.python.org/cpython/rev/313a71664781

New changeset 5993f91598ce by Éric Araujo in branch 'default':
Merge fixes for bpo-9860, bpo-11104/bpo-8688 and bpo-12331 from 3.2
http://hg.python.org/cpython/rev/5993f91598ce

New changeset bea11ce24bb0 by Éric Araujo in branch '2.7':
Let “make patchcheck” work for out-of-dir builds (bpo-9860)
http://hg.python.org/cpython/rev/bea11ce24bb0

FWIW, make patchcheck fails for me as follows:

% make patchcheck
./python /home/trent/hg/cpython-3.2/Tools/scripts/patchcheck.py
Getting the list of files that have been added/changed ... abort: no repository found in '/tmp/cpython-3.2-build' (.hg not found)!
0 files
Fixing whitespace ... 0 files
Fixing C file whitespace ... 0 files
Fixing docs whitespace ... 0 files
Docs modified ... NO
Misc/ACKS updated ... NO
Misc/NEWS updated ... NO
[51233 refs]

patchcheck.py needs to be modified so that it cd's into SRCDIR before running any hg commands.

FYI hg -R path_to_repo works, we don't have to cd.