classification
Title: Lib/symbol.py is out of sync with Grammar/Grammar
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: yselivanov Nosy List: benjamin.peterson, mgedmin, python-dev, r.david.murray, yselivanov
Priority: release blocker Keywords: patch

Created on 2015-05-27 12:51 by mgedmin, last changed 2015-09-02 12:35 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
test_symbol.patch yselivanov, 2015-05-27 15:26 review
Messages (7)
msg244160 - (view) Author: Marius Gedminas (mgedmin) * Date: 2015-05-27 12:51
While investigating https://bitbucket.org/pypa/setuptools/issue/388/install-from-sdist-fails-on-python-350b1 I noticed that Grammar/Grammar changed in 3.5, but Lib/symbol.py wasn't updated.

I'm not familiar with the CPython parser, but I suspect that adding/removing/splitting grammar rules causes the nonterminal symbol IDs to shift, which ought to require an update in symbol.py.

Huh.  Now I see a comment in the file says it is automatically generated, but in that case why wasn't that done when I did 'hg pull -u && make && make install'?  Why is it checked into source control?
msg244168 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-05-27 13:39
It is "automatically generated" in that it isn't hand-written.  On the other hand, it isn't "automatically generated" in the sense of being part of the make process, ./python symbol.py is supposed to be run by hand when it is appropriate.

A bit ago someone wrote tests for keyword.py that among other things made sure we didn't forget to update it when needed.  Someone needs to write a similar test for symbol, it looks like.

Whether or not one or both of these could be/should be incorporated into make (now that we have 'make touch' to deal with the consequences) is a separate question.  As to why it is checked in, we check in almost all the build artifacts previous to the compile stage, so that there is no need to have an already-built python to build python from source.
msg244174 - (view) Author: Roundup Robot (python-dev) Date: 2015-05-27 15:10
New changeset 843fe7e831a8 by Yury Selivanov in branch '3.5':
Issue 24297: Update symbol.py. See also issue 24017.
https://hg.python.org/cpython/rev/843fe7e831a8

New changeset 87509d71653b by Yury Selivanov in branch 'default':
Issue 24297: Update symbol.py. See also issue 24017.
https://hg.python.org/cpython/rev/87509d71653b
msg244177 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-05-27 15:26
Attached is a new unittest to make sure that symbol.py is always updated.  Essentially it's the same test that we have for keywords.py.  Please review.
msg244334 - (view) Author: Roundup Robot (python-dev) Date: 2015-05-28 18:06
New changeset a95d0dd3c2b1 by Yury Selivanov in branch 'default':
Issue 24297: Add a unittest that Lib/symbol.py is in sync with Grammar
https://hg.python.org/cpython/rev/a95d0dd3c2b1
msg244335 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-05-28 18:06
Marius, thanks for reporting the bug and review!
msg249540 - (view) Author: Roundup Robot (python-dev) Date: 2015-09-02 12:35
New changeset bf7ef3bd9a09 by Victor Stinner in branch 'default':
Issue 24297: Fix test_symbol on Windows
https://hg.python.org/cpython/rev/bf7ef3bd9a09
History
Date User Action Args
2015-09-02 12:35:11python-devsetmessages: + msg249540
2015-05-28 18:06:48yselivanovsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-05-28 18:06:37yselivanovsetmessages: + msg244335
2015-05-28 18:06:20python-devsetmessages: + msg244334
2015-05-27 15:26:10yselivanovsetfiles: + test_symbol.patch

assignee: yselivanov
versions: + Python 3.6
keywords: + patch
nosy: + yselivanov

messages: + msg244177
stage: patch review
2015-05-27 15:10:14python-devsetnosy: + python-dev
messages: + msg244174
2015-05-27 14:47:39brett.cannonsetpriority: normal -> release blocker
2015-05-27 13:39:20r.david.murraysetnosy: + r.david.murray, benjamin.peterson
messages: + msg244168
2015-05-27 12:51:14mgedmincreate