Issue15819
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2012-08-30 06:49 by trent, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
readonly_src.patch | trent, 2012-08-30 06:49 | review | ||
hgtouch.diff | doko, 2013-01-27 20:27 | review | ||
unused-flags.patch | martin.panter, 2016-09-12 01:49 | Unneeded code in Py 2 | review |
Pull Requests | |||
---|---|---|---|
URL | Status | Linked | Edit |
PR 552 | closed | dstufft, 2017-03-31 16:36 |
Messages (44) | |||
---|---|---|---|
msg169430 - (view) | Author: Trent Nelson (trent) * | Date: 2012-08-30 06:49 | |
This is somewhat related to Issue #9860. You can't currently build Python out-of-tree when the source directory is readonly. This is because there are a few Parser/pgen/AST/python_asdl.py steps that try and write to $(srcdir). The attached patch fixes this. |
|||
msg169431 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-30 06:59 | |
Instead of using "mkdir -p", it should use $(MKDIR_P) (which actually isn't normally "mkdir -p" :-) Other than that, the patch looks fine. |
|||
msg169432 - (view) | Author: Trent Nelson (trent) * | Date: 2012-08-30 07:19 | |
Hmmmmm. Now I'm confused. Is this what you meant? ------------------- % hg diff diff -r 019a2390b014 Makefile.pre.in --- a/Makefile.pre.in Tue Aug 21 23:41:43 2012 +0000 +++ b/Makefile.pre.in Thu Aug 30 07:16:55 2012 +0000 @@ -43,6 +43,8 @@ GNULD= @GNULD@ +MKDIR_P= @MKDIR_P@ + # Shell used by make (some versions default to the login shell, which is bad) SHELL= /bin/sh @@ -223,8 +225,8 @@ ########################################################################## # Grammar -GRAMMAR_H= $(srcdir)/Include/graminit.h -GRAMMAR_C= $(srcdir)/Python/graminit.c +GRAMMAR_H= Include/graminit.h +GRAMMAR_C= Python/graminit.c GRAMMAR_INPUT= $(srcdir)/Grammar/Grammar @@ -266,9 +268,9 @@ ########################################################################## # AST -AST_H_DIR= $(srcdir)/Include +AST_H_DIR= Include AST_H= $(AST_H_DIR)/Python-ast.h -AST_C_DIR= $(srcdir)/Python +AST_C_DIR= Python AST_C= $(AST_C_DIR)/Python-ast.c AST_ASDL= $(srcdir)/Parser/Python.asdl @@ -605,9 +607,11 @@ Parser/pgenmain.o: $(srcdir)/Include/parsetok.h $(AST_H): $(AST_ASDL) $(ASDLGEN_FILES) + $(MKDIR_P) $(AST_H_DIR) $(ASDLGEN) -h $(AST_H_DIR) $(AST_ASDL) $(AST_C): $(AST_ASDL) $(ASDLGEN_FILES) + $(MKDIR_P) $(AST_C_DIR) $(ASDLGEN) -c $(AST_C_DIR) $(AST_ASDL) Python/compile.o Python/symtable.o Python/ast.o: $(GRAMMAR_H) $(AST_H) diff -r 019a2390b014 configure --- a/configure Tue Aug 21 23:41:43 2012 +0000 +++ b/configure Thu Aug 30 07:16:55 2012 +0000 @@ -658,6 +658,7 @@ GNULD LINKCC LDVERSION +MKDIR_P RUNSHARED INSTSONAME LDLIBRARYDIR @@ -5262,6 +5263,7 @@ HGBRANCH="" fi +MKDIR_P="mkdir -p" DISABLE_ASDLGEN="" # Extract the first word of "python", so it can be a program name with args. ------------------- Given your "which actually isn't normally mkdir -p" comment... my patch doesn't seem right. |
|||
msg169433 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-30 07:28 | |
Am 30.08.12 09:19, schrieb Trent Nelson: > Hmmmmm. Now I'm confused. Is this what you meant? No. Makefile.pre.in *already* has a definition of MKDIR_P, no need to add one. |
|||
msg169434 - (view) | Author: Trent Nelson (trent) * | Date: 2012-08-30 07:30 | |
That's what I figured you meant initially... Until I couldn't find any MKDIR_P definitions, and couldn't get it working without the hack above. I thought it might be an implicit make variable as well, but, not so much. (Tested with BSD make and gmake on FreeBSD.) |
|||
msg169435 - (view) | Author: Petri Lehtinen (petri.lehtinen) * | Date: 2012-08-30 07:34 | |
In 3.3, Makefile.pre.in already defines MKDIR_P, in 3.2 it doesn't. |
|||
msg169437 - (view) | Author: Trent Nelson (trent) * | Date: 2012-08-30 07:47 | |
Ah, I see the MKDIR_P changes in 3.x/configure as well. Objections to backporting to 3.2? Is 2.7 off the table? (Building from a readonly source is handy for Snakebite; saves having a dozen machines needing to use their own hg clone.) |
|||
msg169444 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-30 10:00 | |
Am 30.08.12 09:47, schrieb Trent Nelson: > Objections to backporting to 3.2? > > Is 2.7 off the table? No, adding AC_PROG_MKDIR_P into all of them is fine. |
|||
msg169458 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-08-30 14:32 | |
New changeset 62044cd5b19b by Trent Nelson in branch '3.2': Issue #15819: Fix out-of-tree builds from a readonly source. http://hg.python.org/cpython/rev/62044cd5b19b |
|||
msg169461 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-08-30 14:36 | |
New changeset af36536988fd by Trent Nelson in branch 'default': Block 78809:62044cd5b19b (Issue #15819) from 3.2. http://hg.python.org/cpython/rev/af36536988fd |
|||
msg169462 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-08-30 14:52 | |
New changeset ab6ab44921b2 by Trent Nelson in branch 'default': Issue #15819: Fix out-of-tree builds from a readonly source. http://hg.python.org/cpython/rev/ab6ab44921b2 |
|||
msg169463 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-08-30 14:56 | |
New changeset 2dde5a7439fd by Trent Nelson in branch '2.7': Issue #15819: Fix out-of-tree builds from a readonly source. http://hg.python.org/cpython/rev/2dde5a7439fd |
|||
msg169480 - (view) | Author: Richard Oudkerk (sbt) * | Date: 2012-08-30 18:51 | |
I tried building from out-of-tree with a read-only srcdir, but I found two other files which the build process tries to create in the source directory: $(srcdir)/Objects/typeslots.inc $(srcdir)/Lib/_sysconfigdata.py |
|||
msg169484 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-30 19:09 | |
What I liked about Trent's original bug report is that it didn't just bring up a generic issue (cannot build with read-only source code), but clarified that his concern really was graminit. So I'd like to declare that this (graminit) is the focus of this issue: any other aspects of read-only source trees are out of scope, and should be reported separately. That said: As for typeslots.inc: this shouldn't really be an issue for a released version of Python, since its tarball should include the generated file, and the timestamps should be in an order so that generation is unnecessary. This should be the case for all generated files that are also checked into Mercurial. For checkouts from Mercurial, you should do "make touch" after an update, as this will bring the timestamps in the right order. It seems that graminit is missing from .hgtouch (in which case Trent's patch probably wouldn't have been necessary, except that it also fixes older branches). But that is again a different issue. As for sysconfigdata: this really should be a separate issue. It's more difficult, since installation and packaging needs to pick up the file even if it's not in the source tree. |
|||
msg169486 - (view) | Author: Ned Deily (ned.deily) * | Date: 2012-08-30 19:13 | |
The sysconfigdata problem is already documented in two open issues: Issue14774 and Issue15298. |
|||
msg169489 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-30 19:50 | |
As a further follow-up: the original code (in 67ed8a6905c3, by Neil Schemenauer from 2001), he explicitly changed GRAMMAR_C/H to refer to srcdir, claiming that errors from building them would be ignored. In r87558, for which I can't find a subversion revsision for, Victor Stinner dropped the Makefile "-" marker, making the read-only case fail. In reviewing the patch more, I think it is actually incorrect: it now generates the files into Include, but they aren't considered there since (builddir)/Include is not added to the -I options. So the two options now are: a) revert the patch, and restore Neil's original approach, or b) add Include to the -I options, if srcdir is not "." Reopening for consideration. |
|||
msg169490 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-30 20:02 | |
Reconsidering more: it would be desirable if even an out-of-source-tree compilation wouldn't require Python for bootstrapping; with the committed patch, it does since it will always run ASDLGEN. If the source version of the generated files is fresh enough, they should just be copied to the destination. I'm not quite sure how to achieve that with Make rules. This probably *is* a separate issue (which would become irrelevant if the patch here is reverted). |
|||
msg169534 - (view) | Author: Trent Nelson (trent) * | Date: 2012-08-31 13:19 | |
Well, that escalated quickly :-) I think I should clarify my use case that resulted in raising this bug. 1. Create a fresh hg clone, say, ~/hg/cpython-3.2. 2. Make it readonly: `zfs set readonly=on tank/home/cpython/hg/cpython-3.2` 3. cd /tmp/cpython-3.2-build 4. /home/cpython/hg/cpython-3.2/configure --srcdir=/home/cpython/hg/cpython-3.2/configure So, my patches address that use case. The thing I like about the approach above is that I can set the source directory to readonly immediately -- I don't have to do an intermediate ./configure in that directory in order to generate a Makefile so I can `make touch`. (Side note: I'm almost certain `make touch` wouldn't work for my use case.) Richard: what was your use case? What steps did you take that resulted in getting typeslots.inc and _sysconfigdata.py written to? ....although I just tried an out-of-tree build with 'default' using the exact steps above and got the same error with _sysconfigdata.py. Martin: I think I understand your arguments, and I (think) I agree with them. Especially your last point with regards to always running ASDLGEN. I think I'm +1 on reverting and trying the '-' approach originally used by Neil. |
|||
msg169535 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-31 13:56 | |
Am 31.08.12 15:19, schrieb Trent Nelson: > I think I should clarify my use case that resulted in raising this > bug. > > 1. Create a fresh hg clone, say, ~/hg/cpython-3.2. > 2. Make it readonly The use case is really uncontended, and long-standing. People want to build from CD-ROM sources, from sources mounted via NFS, from sources owned by some other user - there are really many reasons why you want to build in a separate directory, and why you then do not want the original source be clobbered. > Richard: what was your use case? I guess pretty much the same (though perhaps not ZFS). > What steps did you take that > resulted in getting typeslots.inc and _sysconfigdata.py written to? Touch Include/typeslots.h (before making the checkout readonly) to see this problem. On a hg checkout, it really depends on the order in which Mercurial creates the files to either run or not run into one of these issues. You were lucky that the timestamps happened to be in an order were in an order that caused typeslots.inc to be newer than its sources. AFAIK, hg uses alphapbetical order when updating files, so this may have helped. > I think I'm +1 on reverting and trying the '-' approach originally > used by Neil. Please don't. I think Victor dropped the - because it would mask errors from running pgen. pgen might fail, but pgen.stamp is still generated. Then, on a subsequent run, pgen would not run again. Of course, pgen.stamp is a later addition (by me), so this wasn't originally a problem. I think the regular case should be that the time stamps are in the right order in the source tree, so none of the generators should run. But if they happen to be in the wrong order (either because of Mercurial, or because one of the sources was actually modified), then the generators should run and output into the build directory. Since the committed patch is already a step in that direction, I think it now needs to be completed. It's mandatory (IMO) that the generated files are actually used; that the generators are not run if not needed is optional and can be fixed later. |
|||
msg169541 - (view) | Author: Richard Oudkerk (sbt) * | Date: 2012-08-31 14:40 | |
> Richard: what was your use case? Building on the main host and a VM without having to commit and synchronize temporary changes. (The VM has read-only access to the host's repositories). > What steps did you take that resulted in getting typeslots.inc and > _sysconfigdata.py written to? As Martin says it is probably just the timestamps which caused typeslots.inc to be rebuilt. In 3.3 _sysconfigdata.py is rebuilt whenever the python binary is. I suspect a bigger issue is the fact that *.pyc files cannot be written to $(srcdir)/Lib/__pycache__. This means that to complete the build, $(PYTHON_FOR_BUILD) should probably include the -B flag to prevent it from trying to write *.pyc files to a read-only location. But that still leaves you with a python which only works with the -B flag. Maybe library files could be linked/copied to some writable directory early in sys.path. |
|||
msg169542 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-31 14:57 | |
Am 31.08.12 16:40, schrieb Richard Oudkerk: > I suspect a bigger issue is the fact that *.pyc files cannot be > written to $(srcdir)/Lib/__pycache__. This means that to complete > the build, $(PYTHON_FOR_BUILD) should probably include the -B flag to > prevent it from trying to write *.pyc files to a read-only location. Why is that? python will work just fine if it can't write pyc files. > But that still leaves you with a python which only works with the -B > flag. Maybe library files could be linked/copied to some writable > directory early in sys.path. Just for running setup.py? I don't see the need for that. |
|||
msg169544 - (view) | Author: Richard Oudkerk (sbt) * | Date: 2012-08-31 15:32 | |
> Why is that? python will work just fine if it can't write pyc files. When I hacked around the _sysconfigdata.py issue, running the created python produced Fatal Python error: Py_Initialize: Unable to get the locale encoding Traceback (most recent call last): File "/mnt/hgfs/Repos/cpython-dirty/Lib/encodings/__init__.py", line 31, in <module> File "<frozen importlib._bootstrap>", line 1562, in _find_and_load File "<frozen importlib._bootstrap>", line 1529, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 590, in _check_name_wrapper File "<frozen importlib._bootstrap>", line 1027, in load_module File "<frozen importlib._bootstrap>", line 1008, in load_module File "<frozen importlib._bootstrap>", line 566, in module_for_loader_wrapper File "<frozen importlib._bootstrap>", line 858, in _load_module File "<frozen importlib._bootstrap>", line 994, in get_code File "<frozen importlib._bootstrap>", line 1055, in _cache_bytecode File "<frozen importlib._bootstrap>", line 1078, in set_data File "<frozen importlib._bootstrap>", line 128, in _write_atomic OSError: [Errno 5] Input/output error: '/mnt/hgfs/Repos/cpython-dirty/Lib/__pycache__/codecs.cpython-33.pyc.3076609376' Aborted I took this to be a failure to write the bytecompiled file. (A regression?) Running "./python -B" does work correctly for me. |
|||
msg169545 - (view) | Author: Richard Oudkerk (sbt) * | Date: 2012-08-31 15:40 | |
It looks like the code which writes the byte-compiled codes checks for and ignores PermissionError (EACCES, EPERM) and FileExistsError (EEXIST). However, for me it is EIO which is raised. |
|||
msg169546 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-31 15:49 | |
Am 31.08.12 17:32, schrieb Richard Oudkerk: > File "<frozen importlib._bootstrap>", line 128, in _write_atomic > OSError: [Errno 5] Input/output error: '/mnt/hgfs/Repos/cpython-dirty/Lib/__pycache__/codecs.cpython-33.pyc.3076609376' > Aborted > > I took this to be a failure to write the bytecompiled file. (A regression?) Sounds like a regression to me. However, errno 5 is EIO. Normally, you should get PermissionError(errno=13, EACCES). I could understand EROFS as well. This sounds like a bug in the VM software, which really should report EROFS if the volume is read-only, not EIO. You might want to check whether this really is a regression; in any case, I consider Python's reaction reasonable and then your setup as just not supported. However, I could also accept that a work-around is added for treating EIO also as an ignorable error. In any case, this is a separate issue. |
|||
msg169552 - (view) | Author: Eric Snow (eric.snow) * | Date: 2012-08-31 16:18 | |
Doesn't EIO cover pathologies which we don't want to silence? I could see adding EROFS though. |
|||
msg169558 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2012-08-31 16:45 | |
Am 31.08.12 18:18, schrieb Eric Snow: > Doesn't EIO cover pathologies which we don't want to silence? I could see adding EROFS though. [This should *really* be its own issue] Sure, EIO indicates normally something pathological, such as a broken cable to the disk. I'm not sure it needs to be reported, though, at least not when coming out of open(): we are trying to write to the cache, and that failed really bad. However, this is not bad for Python: if we don't have a cache file, we can still continue executing. If we get EIO in the middle of writing, this would be worse. So as a work-around, I could propose to map OSError(EIO) coming out of open into PermissionError(EIO) (inside _bootstrap). But as I said: I'd also be fine with declaring this VM software unsupported. If this work-around is added, we should record the exact software for which it is a work-around, so we can remove it in the future. |
|||
msg169569 - (view) | Author: Richard Oudkerk (sbt) * | Date: 2012-08-31 17:22 | |
I have opened a new issue: http://bugs.python.org/issue15833 It is a regression because in Python 3.2 all failures to open the file for writing were supressed. > But as I said: I'd also be fine with declaring this VM software > unsupported. If this work-around is added, we should record the > exact software for which it is a work-around, so we can remove > it in the future. It is VMware Player 4.0.0. |
|||
msg169595 - (view) | Author: Trent Nelson (trent) * | Date: 2012-09-01 00:03 | |
> Since the committed patch is already a step in that direction, > I think it now needs to be completed. It's mandatory (IMO) > that the generated files are actually used; that the generators > are not run if not needed is optional and can be fixed later. Martin: I'll work on another patch over the next few days and post it here for review. It seems like your comments regarding our options are still valid (despite this thread getting sidetracked): continue with the premise that all generated files get generated in the build directory, and make sure that build directory gets used during the build (i.e. -I./Include). I'll start with that and see how I get on. |
|||
msg173038 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-10-16 12:57 | |
New changeset 617591e7d708 by Trent Nelson in branch '3.2': Issue #15819: additional fixes for out-of-tree builds from read-only src. http://hg.python.org/cpython/rev/617591e7d708 New changeset abb00e23681a by Trent Nelson in branch '3.3': Merge issue #15819: additional fixes for out-of-tree builds from read-only src. http://hg.python.org/cpython/rev/abb00e23681a New changeset 3420fbd87646 by Trent Nelson in branch 'default': Merge issue #15819: additional fixes for out-of-tree builds from read-only src. http://hg.python.org/cpython/rev/3420fbd87646 |
|||
msg173041 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-10-16 13:43 | |
New changeset e0a2b14a3cf9 by Trent Nelson in branch '2.7': Issue #15819: additional fixes for out-of-tree builds from read-only src. http://hg.python.org/cpython/rev/e0a2b14a3cf9 |
|||
msg173042 - (view) | Author: Trent Nelson (trent) * | Date: 2012-10-16 13:52 | |
Things should be looking a lot better now across the board. I just committed fixes where appropriate to 2.7, 3.2, 3.3 and 3.x for this issue and two related issues: http://bugs.python.org/issue15298 http://bugs.python.org/issue15833 The only thing that stands out now for 3.[23x] is $(srcdir)/Lib/$(PLATDIR). Do we need a patch along the following lines? % hg diff diff -r 617591e7d708 Makefile.pre.in --- a/Makefile.pre.in Tue Oct 16 08:41:32 2012 -0400 +++ b/Makefile.pre.in Tue Oct 16 09:46:53 2012 -0400 @@ -947,7 +947,7 @@ multiprocessing multiprocessing/dummy \ unittest unittest/test \ curses pydoc_data $(MACHDEPS) -libinstall: build_all $(srcdir)/Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c +libinstall: build_all Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c @for i in $(SCRIPTDIR) $(LIBDEST); \ do \ if test ! -d $(DESTDIR)$$i; then \ @@ -1031,14 +1031,14 @@ ./$(BUILDPYTHON) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt # Create the PLATDIR source directory, if one wasn't distributed.. -$(srcdir)/Lib/$(PLATDIR): - mkdir $(srcdir)/Lib/$(PLATDIR) - cp $(srcdir)/Lib/plat-generic/regen $(srcdir)/Lib/$(PLATDIR)/regen +Lib/$(PLATDIR): + mkdir Lib/$(PLATDIR) + cp $(srcdir)/Lib/plat-generic/regen Lib/$(PLATDIR)/regen export PATH; PATH="`pwd`:$$PATH"; \ export PYTHONPATH; PYTHONPATH="`pwd`/Lib"; \ export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`"; \ export EXE; EXE="$(BUILDEXE)"; \ - cd $(srcdir)/Lib/$(PLATDIR); $(RUNSHARED) ./regen + cd Lib/$(PLATDIR); $(RUNSHARED) ./regen python-config: $(srcdir)/Misc/python-config.in # Substitution happens here, as the completely-expanded BINDIR |
|||
msg173046 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-10-16 15:11 | |
New changeset c35db4960d1c by Trent Nelson in branch '2.7': Issue #15819: tweak logic in previous commit (e0a2b14a3cf9). http://hg.python.org/cpython/rev/c35db4960d1c New changeset 5707c8a36cc0 by Trent Nelson in branch '3.2': Issue #15819: tweak logic in previous commit (617591e7d708). http://hg.python.org/cpython/rev/5707c8a36cc0 New changeset 85863c4a93db by Trent Nelson in branch '3.3': Merge issue #15819: tweak logic in previous commit. http://hg.python.org/cpython/rev/85863c4a93db New changeset d282018bff10 by Trent Nelson in branch 'default': Merge issue #15819: tweak logic in previous commit. http://hg.python.org/cpython/rev/d282018bff10 |
|||
msg173144 - (view) | Author: Roundup Robot (python-dev) | Date: 2012-10-17 08:49 | |
New changeset 7e879a453bb3 by Trent Nelson in branch '2.7': Issue #15819: use standard autoconf preset output variables. http://hg.python.org/cpython/rev/7e879a453bb3 New changeset 65b3c41052b6 by Trent Nelson in branch '3.2': Issue #15819: use standard autoconf preset output variables. http://hg.python.org/cpython/rev/65b3c41052b6 New changeset b78e97b84f35 by Trent Nelson in branch '3.3': Issue #15819: use standard autoconf preset output variables. http://hg.python.org/cpython/rev/b78e97b84f35 New changeset f9eb516bea88 by Trent Nelson in branch 'default': Issue #15819: use standard autoconf preset output variables. http://hg.python.org/cpython/rev/f9eb516bea88 |
|||
msg180788 - (view) | Author: Matthias Klose (doko) * | Date: 2013-01-27 20:27 | |
just saw the comment about .hgtouch in http://bugs.python.org/issue15819#msg169484 attaching a patch for it here (as proposed on python-committers). |
|||
msg181177 - (view) | Author: Roumen Petrov (rpetrov) * | Date: 2013-02-02 16:33 | |
To update .hgtouch is not enough. Grammar, AST and importlib should left in source tree as before. |
|||
msg203348 - (view) | Author: Kubilay Kocak (koobs) | Date: 2013-11-19 10:08 | |
I'm not sure how I missed this, but our FreeBSD ports that build out-of-tree required adding a local patchthat reverts the Makefile.pre.in change in ab6ab44921b2 when we updated each port to the latest releases of 2.7, 3.2 and 3.3: https://svnweb.freebsd.org/ports?view=revision&revision=318353 Without it, builds fail with: cc -c -fno-strict-aliasing -O2 -pipe -fno-strict-aliasing -DNDEBUG -I. -IInclude -I./../Include -I/usr/local/include -fPIC -DPy_BUILD_CORE -o Python/Python-ast.o Python/Python-ast.c cc: Python/Python-ast.c: No such file or directory cc: No input files specified *** [Python/Python-ast.o] Error code 1 Stop in /freebsd/ports/lang/python27/work/Python-2.7.6/portbld.shared. I also note Trents msg173160 in issue #15298, which may be related?, but seems only resolved for OSX. What needs to be done to remove the need for this patch? |
|||
msg275751 - (view) | Author: Martin Panter (martin.panter) * | Date: 2016-09-11 09:08 | |
Perhaps there should be a comment explaining that you can’t write to $(srcdir). Or even a buildbot that does an out-of-tree build. Every now and then I seem to be defending against patches that want to change targets back to $(srcdir). Also, Trent’s last change, revision 65b3c41052b6, does not seem to work. Apparently $abs_srcdir etc are only valid when substituted in a makefile, and are not valid in the configure script: <https://lists.gnu.org/archive/html/autoconf/2010-07/msg00021.html>. I will try to post a patch that addresses this to Issue 28066. |
|||
msg275768 - (view) | Author: Kubilay Kocak (koobs) | Date: 2016-09-11 10:32 | |
@Martin Maybe Zach (nosey'd) can help us get an out-of-tree builder set up? If this is relevant for 3.5+, versions should be updated accordingly. |
|||
msg275910 - (view) | Author: Martin Panter (martin.panter) * | Date: 2016-09-12 01:49 | |
I’m not sure what the overall status of this bug is, so I will leave the versions as they are. Are the three comments from 2013 relevant, and is there anything I can do to help? The Python 2 code already unconditionally adds -IInclude (see Issue 786737), and does not use BASECPPFLAGS. I think it would better to remove Trent’s code, as in unused-flags.patch. Python 3 also has this unconditional -IInclude, so maybe we can eliminate one of them in Python 3? gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -IObjects -IInclude -IPython -I. -IInclude -I../Include -DPy_BUILD_CORE -o Python/Python-ast.o Python/Python-ast.c |
|||
msg275948 - (view) | Author: Martin Panter (martin.panter) * | Date: 2016-09-12 04:24 | |
Matthias seems to have already applied his patch in Python 3.3 and 3.4+: revision f2cc3d8b88bb. Roumen: Is your problem still relevant? If so, perhaps open a separate bug to elaborate. Koobs: Is your problem with finding Python/Python-ast.c still relevant? Is the $(ASDLGEN) makefile rule being run? I tried building both Python 2 and 3 outside the source tree, with Gnu and BSD Make. In all cases, if I hack the timestamps to force $(ASDLGEN) to run and remove the original source Python/Python-ast.c, it is correctly generated and compiled in the build directory. If I hack the timestamps the other way so that $(ASDLGEN) is not run, the file from the source directory is compiled into the build directory. So everything seems to be working reasonably well. |
|||
msg275995 - (view) | Author: Kubilay Kocak (koobs) | Date: 2016-09-12 08:32 | |
@Martin tldr; That was three years ago so I don't know if the issue remains. Perhaps a new out of tree builder will help answer that question. We(FreeBSD) gave up^W switched away from out of tree builds due to lack of support getting issues fixed and the frequency at which commits would alternately break in tree builds then out of tree builds. Unfortunately that also lost us the flexibility to create dual static/shared builds. The relevant downstream commit log: https://svnweb.freebsd.org/ports?view=revision&revision=350610 The patch we added (reverting 2dde5a7439fd) that was mentioned in msg203348 was: https://svnweb.freebsd.org/ports/head/lang/python27/files/patch-Makefile.pre.in?view=markup&pathrev=318353 If the current code is the same as per the 2dde5a7439fd change, then I'd guess it would still be an issue. |
|||
msg276260 - (view) | Author: Roundup Robot (python-dev) | Date: 2016-09-13 12:26 | |
New changeset 9cabcb4411ac by Martin Panter in branch '2.7': Issue #15819: Drop dead code; -IInclude already added unconditionally https://hg.python.org/cpython/rev/9cabcb4411ac |
|||
msg276265 - (view) | Author: Roundup Robot (python-dev) | Date: 2016-09-13 13:29 | |
New changeset 36550e4f9b4c by Martin Panter in branch '3.5': Issue #15819: Remove old unconditional -IInclude option https://hg.python.org/cpython/rev/36550e4f9b4c New changeset a90daae58323 by Martin Panter in branch '3.6': Issue #15819: Merge include search from 3.5 into 3.6 https://hg.python.org/cpython/rev/a90daae58323 New changeset 2ac00ff072b8 by Martin Panter in branch 'default': Issue #15819: Merge include search from 3.6 https://hg.python.org/cpython/rev/2ac00ff072b8 |
|||
msg276667 - (view) | Author: Martin Panter (martin.panter) * | Date: 2016-09-16 02:48 | |
Just to clarify, the current status is that the revision that Koobs identified is still in place, meaning that the AST files are regenerated into the build tree, not the source tree. I don’t think there is much else to do for this bug, unless Koobs can provide more info why the Free BSD build could not find Python/Python-ast.c, or Roumen can get back with more info. Closing for the moment, but I am happy to help if I can resolve any problems that remain. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:57:35 | admin | set | github: 60023 |
2017-03-31 16:36:17 | dstufft | set | pull_requests: + pull_request914 |
2016-09-16 02:48:55 | martin.panter | set | status: open -> closed resolution: fixed messages: + msg276667 stage: patch review -> resolved |
2016-09-13 13:29:29 | python-dev | set | messages: + msg276265 |
2016-09-13 12:26:16 | python-dev | set | messages: + msg276260 |
2016-09-12 09:41:11 | petri.lehtinen | set | nosy:
- petri.lehtinen |
2016-09-12 08:32:25 | koobs | set | messages: + msg275995 |
2016-09-12 04:24:06 | martin.panter | set | messages: + msg275948 |
2016-09-12 01:49:51 | martin.panter | set | files:
+ unused-flags.patch messages: + msg275910 stage: patch review |
2016-09-11 10:32:40 | koobs | set | nosy:
+ zach.ware messages: + msg275768 |
2016-09-11 09:08:20 | martin.panter | set | nosy:
+ martin.panter dependencies: + [Patch] Fix the ability to cross compile Python when doing a rebuild of importlib.h messages: + msg275751 |
2014-03-04 08:19:25 | gregory.p.smith | set | nosy:
+ gregory.p.smith |
2013-11-19 10:08:05 | koobs | set | nosy:
+ koobs messages: + msg203348 |
2013-05-22 20:29:47 | ned.deily | link | issue17649 superseder |
2013-02-02 16:33:59 | rpetrov | set | nosy:
+ rpetrov messages: + msg181177 |
2013-01-27 20:27:51 | doko | set | files:
+ hgtouch.diff nosy: + doko messages: + msg180788 |
2012-10-17 08:49:44 | python-dev | set | messages: + msg173144 |
2012-10-16 15:11:25 | python-dev | set | messages: + msg173046 |
2012-10-16 13:52:14 | trent | set | nosy:
+ pitrou messages: + msg173042 |
2012-10-16 13:43:19 | python-dev | set | messages: + msg173041 |
2012-10-16 12:57:24 | python-dev | set | messages: + msg173038 |
2012-10-14 08:44:09 | vstinner | set | nosy:
+ vstinner |
2012-09-04 07:47:54 | Arfrever | set | nosy:
+ Arfrever |
2012-09-01 00:03:14 | trent | set | messages: + msg169595 |
2012-08-31 17:22:51 | sbt | set | messages: + msg169569 |
2012-08-31 16:45:43 | loewis | set | messages: + msg169558 |
2012-08-31 16:18:03 | eric.snow | set | nosy:
+ eric.snow messages: + msg169552 |
2012-08-31 15:49:06 | loewis | set | messages: + msg169546 |
2012-08-31 15:40:05 | sbt | set | messages: + msg169545 |
2012-08-31 15:32:08 | sbt | set | messages: + msg169544 |
2012-08-31 14:57:22 | loewis | set | messages: + msg169542 |
2012-08-31 14:40:37 | sbt | set | messages: + msg169541 |
2012-08-31 13:56:30 | loewis | set | messages: + msg169535 |
2012-08-31 13:19:09 | trent | set | messages: + msg169534 |
2012-08-31 10:27:29 | asvetlov | set | stage: resolved -> (no value) |
2012-08-30 20:02:51 | loewis | set | messages: + msg169490 |
2012-08-30 19:50:49 | loewis | set | status: closed -> open messages: + msg169489 |
2012-08-30 19:13:47 | ned.deily | set | nosy:
+ ned.deily messages: + msg169486 |
2012-08-30 19:09:33 | loewis | set | messages: + msg169484 |
2012-08-30 18:51:09 | sbt | set | nosy:
+ sbt messages: + msg169480 |
2012-08-30 14:57:20 | trent | set | status: open -> closed stage: patch review -> resolved |
2012-08-30 14:56:23 | python-dev | set | messages: + msg169463 |
2012-08-30 14:52:53 | python-dev | set | messages: + msg169462 |
2012-08-30 14:36:37 | python-dev | set | messages: + msg169461 |
2012-08-30 14:32:13 | python-dev | set | nosy:
+ python-dev messages: + msg169458 |
2012-08-30 10:00:06 | loewis | set | messages: + msg169444 |
2012-08-30 07:47:23 | trent | set | messages: + msg169437 |
2012-08-30 07:34:06 | petri.lehtinen | set | nosy:
+ petri.lehtinen messages: + msg169435 |
2012-08-30 07:30:09 | trent | set | messages: + msg169434 |
2012-08-30 07:28:15 | loewis | set | messages:
+ msg169433 title: Unable to build Python out-of-tree when source tree is readonly. -> Unable to build Python out-of-tree when source tree is readonly. |
2012-08-30 07:19:41 | trent | set | messages: + msg169432 |
2012-08-30 06:59:27 | loewis | set | nosy:
+ loewis messages: + msg169431 |
2012-08-30 06:49:31 | trent | set | stage: needs patch -> patch review |
2012-08-30 06:49:21 | trent | create |