classification
Title: Auto-generate Lib/keyword.py
Type: Stage: resolved
Components: Build Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: cheryl.sabella, gvanrossum, nanjekyejoannah, pablogsal, serhiy.storchaka, vstinner, zach.ware
Priority: normal Keywords: easy, patch

Created on 2019-02-28 04:11 by gvanrossum, last changed 2019-03-25 22:01 by pablogsal. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12456 merged pablogsal, 2019-03-20 00:36
Messages (10)
msg336797 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-02-28 04:11
The stdib keyword.py module must be regenerated after adding/removing keywords from the grammar. While this is rare, we now generate everything else derived from the grammar. Hopefully someone can add some rules to the Makefile to auto-generate this one too when regen-grammar is run.

This is probably an easy project for a beginning contributor.
msg336799 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2019-02-28 04:35
Lib/keyword.py already regenerates itself from Python/graminit.c when run as a script, but there should be a `regen-keyword.py` target added to Makefile.pre.in and included as a dependency of the `regen-all` target.  Agreed that this is a good starting issue, particularly for anyone wanting to get more familiar with Makefiles.

See also issue30638.
msg336835 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2019-02-28 14:46
I am working on this. Shouldn't there be a PR by tomorrow from me. Someone else can take it on.
msg336836 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-28 15:01
Do not haste with this. After implementing pgen in Python, Lib/keyword.py can be generated directly by pgen.
msg336838 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2019-02-28 15:12
Noted. Releasing this in favor of when pgen is implemented in Python to solve this.
msg336839 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-02-28 15:35
A similar thing seems to be the list of keywords in Lib/pydoc.py. The recipe says

    # CAUTION: if you change one of these dictionaries, be sure to adapt the
    #          list of needed labels in Doc/tools/extensions/pyspecific.py and
    #          regenerate the pydoc_data/topics.py file by running
    #              make pydoc-topics
    #          in Doc/ and copying the output file into the Lib/ directory.
msg336840 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-02-28 15:36
(However that doesn't seem to work. Anyway, there's a list of all keywords there.)
msg336842 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-02-28 15:47
Hum, it seems the right solution was to arrange for async and await to appear in the list of keywords despite their special status. Nevertheless in theory there's something that can be done here to auto-regenerate the list derived from the Grammar.
msg336844 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-02-28 15:51
> Do not haste with this. After implementing pgen in Python, Lib/keyword.py can be generated directly by pgen.

That's bpo-35808 and PR 11814 which is under review.

Anyway, at the end, it would be nice if Lib/keyword.py would be regenerated by "make regen-all". Currently, "regen-all" doesn't regenerate everything. For example, configure is not regenerated.
msg338825 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-03-25 22:01
New changeset 91759d98015e1d6d5e1367cff60592ab548e7806 by Pablo Galindo in branch 'master':
bpo-36143: Regenerate Lib/keyword.py from the Grammar and Tokens file using pgen (GH-12456)
https://github.com/python/cpython/commit/91759d98015e1d6d5e1367cff60592ab548e7806
History
Date User Action Args
2019-03-25 22:01:32pablogsalsetnosy: + pablogsal
messages: + msg338825
2019-03-25 22:01:24pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-03-20 00:36:35pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request12409
2019-02-28 15:51:25vstinnersetnosy: + vstinner
messages: + msg336844
2019-02-28 15:47:55gvanrossumsetmessages: + msg336842
2019-02-28 15:36:49gvanrossumsetmessages: + msg336840
2019-02-28 15:35:52gvanrossumsetmessages: + msg336839
2019-02-28 15:12:50nanjekyejoannahsetmessages: + msg336838
2019-02-28 15:01:49serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg336836
2019-02-28 14:46:04nanjekyejoannahsetnosy: + nanjekyejoannah
messages: + msg336835
2019-02-28 04:44:29xtreaksetnosy: + cheryl.sabella
2019-02-28 04:35:11zach.waresetkeywords: + easy
nosy: + zach.ware
messages: + msg336799

2019-02-28 04:14:29Arfreversettitle: AUto-generate Lib/keyword.py -> Auto-generate Lib/keyword.py
2019-02-28 04:11:10gvanrossumcreate