Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE needs syntax highlighting for async and await #73892

Closed
DavidEFrancoG mannequin opened this issue Mar 3, 2017 · 15 comments
Closed

IDLE needs syntax highlighting for async and await #73892

DavidEFrancoG mannequin opened this issue Mar 3, 2017 · 15 comments
Assignees
Labels
easy topic-IDLE type-feature A feature request or enhancement

Comments

@DavidEFrancoG
Copy link
Mannequin

DavidEFrancoG mannequin commented Mar 3, 2017

BPO 29706
Nosy @terryjreedy, @vadmium, @1st1, @miss-islington
PRs
  • bpo-29706: Test that IDLE colors async/await as keywords. #6846
  • [3.7] bpo-29706: Test that IDLE colors async/await as keywords. (GH-6846) #6867
  • [3.6] bpo-29706: Test that IDLE colors async/await as keywords. (GH-6846) #6868
  • [3.6] bpo-29706: IDLE now colors async and await as keywords in 3.6. #6879
  • Files
  • no syntax highlighting.png: sample screenshot of the current behavior
  • colorize_async.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2018-05-16.01:38:52.201>
    created_at = <Date 2017-03-03.03:43:57.355>
    labels = ['easy', 'expert-IDLE', 'type-feature']
    title = 'IDLE needs syntax highlighting for async and await'
    updated_at = <Date 2018-05-16.01:38:52.200>
    user = 'https://bugs.python.org/DavidEFrancoG'

    bugs.python.org fields:

    activity = <Date 2018-05-16.01:38:52.200>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2018-05-16.01:38:52.201>
    closer = 'terry.reedy'
    components = ['IDLE']
    creation = <Date 2017-03-03.03:43:57.355>
    creator = 'David E. Franco G.'
    dependencies = []
    files = ['46692', '46693']
    hgrepos = []
    issue_num = 29706
    keywords = ['patch', 'easy']
    message_count = 15.0
    messages = ['288849', '288859', '288860', '288868', '288879', '288902', '288920', '288935', '288940', '288945', '316613', '316690', '316711', '316722', '316746']
    nosy_count = 5.0
    nosy_names = ['terry.reedy', 'martin.panter', 'yselivanov', 'David E. Franco G.', 'miss-islington']
    pr_nums = ['6846', '6867', '6868', '6879']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue29706'
    versions = ['Python 3.6']

    @DavidEFrancoG
    Copy link
    Mannequin Author

    DavidEFrancoG mannequin commented Mar 3, 2017

    Well, this is pretty self explanatory, when playing with this new features of async and await (https://docs.python.org/3.5/whatsnew/3.5.html#new-features) I found to me surprise that there is no syntax highlighting for it in the IDLE for py3.5 and also for py3.6

    So I humbly ask for its addition. Thanks

    @DavidEFrancoG DavidEFrancoG mannequin added the type-bug An unexpected behavior, bug, or error label Mar 3, 2017
    @DavidEFrancoG DavidEFrancoG mannequin added the topic-IDLE label Mar 3, 2017
    @terryjreedy
    Copy link
    Member

    I presume you would like 'async' and 'await' highlighted as keywords. However, IDLE takes its definition of 'keyword' from keyword.kwlist. 'Async' and 'await' are currently not on that list as they are not yet keywords.

    >> async = 1
    >> await = 2

    According to https://www.python.org/dev/peps/pep-0492/#transition-plan, the intention was (is?) to make them keywords in 3.7. As of Feb 11, that had not happened yet. If and when it does, this issue will be taken care of.

    I may consider adding a special context sensitive case for 3.6, if the colorizer code makes it easily possible. Care is needed since it would be a mistake to mark them as keywords in the above statements.

    The difficulty is that colorizer uses regexes and a bit of context sensitive code, while python is using a full grammar parse. I believe that either at the beginning of a line and 'async' followed by 'for' or 'with' or 'await' not followed by certain punctuation ('.', ',', or '=') should be treated as a keyword.

    Yury, does the rule above look about right? 'Await' seems trickier than 'async'.

    I think it may be possible to add regexes that are not literal words to the kwlist. If so, adding 'async +def ', 'async +for ', 'async +with ', and 'await +[^.,=]' and not worrying about 'beginning or line' or tabs (instead of spaces) between would be easy and probably good enough.

    @terryjreedy terryjreedy added 3.7 (EOL) end of life type-feature A feature request or enhancement and removed type-bug An unexpected behavior, bug, or error labels Mar 3, 2017
    @1st1
    Copy link
    Member

    1st1 commented Mar 3, 2017

    According to https://www.python.org/dev/peps/pep-0492/#transition-plan, the intention was (is?) to make them keywords in 3.7. As of Feb 11, that had not happened yet. If and when it does, this issue will be taken care of.

    I plan to do that in the next couple of weeks.

    I may consider adding a special context sensitive case for 3.6, if the colorizer code makes it easily possible. Care is needed since it would be a mistake to mark them as keywords in the above statements.

    If possible I would make 'async' and 'await' to be always highlighted. Even if a user writes <3.7 code and uses 'async' as a variable, it would still be great to let the user know that there is something wrong with this variable.

    @terryjreedy
    Copy link
    Member

    Always coloring 'async' and 'await' is trivial (augment expression and add test line) and acceptable to me.

    I am a git neophyte and am waiting for workflow development to settle down before working on new setup for git. If you want to submit the change for 3.6.1, go ahead. (But leave issue open.) Otherwise, I will get it into 3.6.2 (and 3.7). Patch is against 3.6 idlelib.colorize, but 3.7 is identical.

    The test is to run colorizer.py as main module (python -m idlelib.colorize, or F5 in IDLE editor). Click 'Test' button and check that 'async' and 'await' have same color as other keywords. This cannot affect anything in test suite; colorizer is only used by turtledemo, which is not tested, and which will not notice anyway.

    @vadmium
    Copy link
    Member

    vadmium commented Mar 3, 2017

    See also bpo-26264 about the “keyword” module

    @DavidEFrancoG
    Copy link
    Mannequin Author

    DavidEFrancoG mannequin commented Mar 3, 2017

    I presume you would like 'async' and 'await' highlighted as keywords.

    Yes.
    Some others tools like IPython and the Spider IDE that come with the Anaconda package already highlighted them as keywords, even if you can use them as normal variables.

    The test is to run colorizer.py as main module (python -m >idlelib.colorize, or F5 in IDLE editor). Click 'Test' button and check >that 'async' and 'await' have same color as other keywords. This >cannot affect anything in test suite; colorizer is only used by >turtledemo, which is not tested, and which will not notice anyway.

    super, what I have to do to put the patch so I can test it?

    @terryjreedy
    Copy link
    Member

    David, the easiest thing to do would be to copy and paste the following
    + ['async', 'await']
    into this line of colorizer.py (ColorDelegator.py in 3.5)
    kw = r"\b" + any("KEYWORD", keyword.kwlist) + r"\b"
    about line 17, to get
    kw = r"\b" + any("KEYWORD", keyword.kwlist + ['async', 'await']) + r"\b"

    @DavidEFrancoG
    Copy link
    Mannequin Author

    DavidEFrancoG mannequin commented Mar 3, 2017

    Ok, Done.
    that work in both 3.5 and 3.6
    I also did the

    python -m idlelib.ColorDelegator
    

    and

    python -m idlelib.colorizer
    

    and they turn ok

    I also open those modules in their respective idle and run them, ColorDelegator work ok but colorizer throw me this error

        Traceback (most recent call last):
          File "C:\Anaconda3\Lib\idlelib\colorizer.py", line 279, in <module>
            verbosity=2, exit=False)
          File "C:\Anaconda3\lib\unittest\main.py", line 63, in __init__
            self.module = __import__(module)
          File "C:\Anaconda3\lib\idlelib\idle_test\test_colorizer.py", line 8, in <module>
            from test.support import requires
        ImportError: bad magic number in 'test': b'\x03\xf3\r\n'
        >>>

    @terryjreedy
    Copy link
    Member

    If 'import test' fails that way, then there is a problem either with Anaconda or your installation. You might try deleting .../Lib/test/pycache/init*.pyc.

    @DavidEFrancoG
    Copy link
    Mannequin Author

    DavidEFrancoG mannequin commented Mar 3, 2017

    I found the problem, it was a test.pyc file that was in my personal folder... once deleted it work perfect

    the build-in test module should have some other name...

    @terryjreedy
    Copy link
    Member

    PR 6846 adds the htest line to all versions (no news). When merged, I will augment the keyword list and add a news blurb for 3.6.

    @terryjreedy terryjreedy removed the 3.7 (EOL) end of life label May 15, 2018
    @terryjreedy
    Copy link
    Member

    New changeset 389a48e by Terry Jan Reedy in branch 'master':
    bpo-29706: Test that IDLE colors async/await as keywords. (GH-6846)
    389a48e

    @miss-islington
    Copy link
    Contributor

    New changeset 8717cfe by Miss Islington (bot) in branch '3.7':
    bpo-29706: Test that IDLE colors async/await as keywords. (GH-6846)
    8717cfe

    @miss-islington
    Copy link
    Contributor

    New changeset 7c59a33 by Miss Islington (bot) in branch '3.6':
    bpo-29706: Test that IDLE colors async/await as keywords. (GH-6846)
    7c59a33

    @terryjreedy
    Copy link
    Member

    New changeset 1b0d65f by Terry Jan Reedy in branch '3.6':
    [3.6] bpo-29706: IDLE now colors async and await as keywords in 3.6. (bpo-6879)
    1b0d65f

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    easy topic-IDLE type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants