Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auto-generate Lib/keyword.py #80324

Closed
gvanrossum opened this issue Feb 28, 2019 · 11 comments
Closed

Auto-generate Lib/keyword.py #80324

gvanrossum opened this issue Feb 28, 2019 · 11 comments
Labels
3.8 only security fixes build The build process and cross-build easy

Comments

@gvanrossum
Copy link
Member

BPO 36143
Nosy @vstinner, @zware, @serhiy-storchaka, @csabella, @pablogsal, @nanjekyejoannah
PRs
  • bpo-36143: Regenerate Lib/keyword.py from the Grammar and Tokens file using pgen #12456
  • bpo-36143: make regen-all now also runs regen-keyword #24245
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2019-03-25.22:01:24.534>
created_at = <Date 2019-02-28.04:11:10.157>
labels = ['easy', 'build', '3.8']
title = 'Auto-generate Lib/keyword.py'
updated_at = <Date 2021-01-18.20:59:19.413>
user = 'https://github.com/gvanrossum'

    bugs.python.org fields:

    activity = <Date 2021-01-18.20:59:19.413>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2019-03-25.22:01:24.534>
closer = 'pablogsal'
components = ['Build']
creation = <Date 2019-02-28.04:11:10.157>
creator = 'gvanrossum'
dependencies = []
files = []
hgrepos = []
issue_num = 36143
keywords = ['patch', 'easy']
message_count = 11.0
messages = ['336797', '336799', '336835', '336836', '336838', '336839', '336840', '336842', '336844', '338825', '385224']
nosy_count = 6.0
nosy_names = ['vstinner', 'zach.ware', 'serhiy.storchaka', 'cheryl.sabella', 'pablogsal', 'nanjekyejoannah']
pr_nums = ['12456', '24245']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36143'
versions = ['Python 3.8']
    @gvanrossum
    Copy link
    Member Author

    The stdib keyword.py module must be regenerated after adding/removing keywords from the grammar. While this is rare, we now generate everything else derived from the grammar. Hopefully someone can add some rules to the Makefile to auto-generate this one too when regen-grammar is run.

    This is probably an easy project for a beginning contributor.

    @gvanrossum gvanrossum added 3.8 only security fixes build The build process and cross-build labels Feb 28, 2019
    @Arfrever Arfrever mannequin changed the title AUto-generate Lib/keyword.py Auto-generate Lib/keyword.py Feb 28, 2019
    @zware
    Copy link
    Member

    zware commented Feb 28, 2019

    Lib/keyword.py already regenerates itself from Python/graminit.c when run as a script, but there should be a regen-keyword.py target added to Makefile.pre.in and included as a dependency of the regen-all target. Agreed that this is a good starting issue, particularly for anyone wanting to get more familiar with Makefiles.

    See also bpo-30638.

    @zware zware added the easy label Feb 28, 2019
    @nanjekyejoannah
    Copy link
    Member

    I am working on this. Shouldn't there be a PR by tomorrow from me. Someone else can take it on.

    @serhiy-storchaka
    Copy link
    Member

    Do not haste with this. After implementing pgen in Python, Lib/keyword.py can be generated directly by pgen.

    @nanjekyejoannah
    Copy link
    Member

    Noted. Releasing this in favor of when pgen is implemented in Python to solve this.

    @gvanrossum
    Copy link
    Member Author

    A similar thing seems to be the list of keywords in Lib/pydoc.py. The recipe says

    # CAUTION: if you change one of these dictionaries, be sure to adapt the
#          list of needed labels in [Doc/tools/extensions/pyspecific.py](https://github.com/python/cpython/blob/main/Doc/tools/extensions/pyspecific.py) and
#          regenerate the pydoc_data/topics.py file by running
#              make pydoc-topics
#          in Doc/ and copying the output file into the Lib/ directory.

    @gvanrossum
    Copy link
    Member Author

    (However that doesn't seem to work. Anyway, there's a list of all keywords there.)

    @gvanrossum
    Copy link
    Member Author

    Hum, it seems the right solution was to arrange for async and await to appear in the list of keywords despite their special status. Nevertheless in theory there's something that can be done here to auto-regenerate the list derived from the Grammar.

    @vstinner
    Copy link
    Member

    Do not haste with this. After implementing pgen in Python, Lib/keyword.py can be generated directly by pgen.

    That's bpo-35808 and PR 11814 which is under review.

    Anyway, at the end, it would be nice if Lib/keyword.py would be regenerated by "make regen-all". Currently, "regen-all" doesn't regenerate everything. For example, configure is not regenerated.

    @pablogsal
    Copy link
    Member

    New changeset 91759d9 by Pablo Galindo in branch 'master':
    bpo-36143: Regenerate Lib/keyword.py from the Grammar and Tokens file using pgen (GH-12456)
    91759d9

    @vstinner
    Copy link
    Member

    New changeset 6a809fa by Victor Stinner in branch 'master':
    bpo-36143: make regen-all now also runs regen-keyword (GH-24245)
    6a809fa

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes build The build process and cross-build easy
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    6 participants
    @vstinner @gvanrossum @serhiy-storchaka @zware @pablogsal @nanjekyejoannah