classification
Title: IDLE: Include keywords in module-level autocomplete list
Type: enhancement Stage: resolved
Components: IDLE Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: cheryl.sabella, miss-islington, rhettinger, taleinat, terry.reedy, xtreak
Priority: normal Keywords: patch

Created on 2019-08-05 18:03 by xtreak, last changed 2020-07-09 23:47 by terry.reedy. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 15138 merged terry.reedy, 2019-08-05 20:24
PR 21423 merged miss-islington, 2020-07-09 22:09
PR 21424 merged miss-islington, 2020-07-09 22:09
Messages (16)
msg349061 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-08-05 18:03
Currently, the basic repl for python provides keywords as part of autocompletion but IDLE doesn't provide them. I was trying to build an async repl on top of IDLE to support top level await statements as part of IDLE since "python -m asyncio" doesn't provide a good repl and found during usage keywords like async/await being part of autocomplete to provide a good experience like the basic repl to type faster. I couldn't find any old issues with search around why keywords were excluded so I thought of filing a new one for this suggestion.
msg349064 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-05 18:29
The global completion list (i.e. when not completing a file name or object attribute) is already full of all the built-ins, imported modules and variables.  So IMO we'd need a good reason to add yet more options into the completions list.

Personally, I don't think that adding all of the keywords to that list would be helpful: They are all short words and most of them must be memorized anyways to work with Python.

For instance, I don't recall this being brought up by those who often teach newcomers with IDLE, such as Raymond Hettinger, when discussing what's missing in IDLE. I'd be happy to get more input from them on this.
msg349065 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-05 18:30
To be clear, I'm currently -1 on this suggestion.
msg349067 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-05 20:16
If keywords are included when the REPL has tab completions (which Windows doesn't), then it is plausible that IDLE should.  It could be considered part of 'Shell should (mostly) imitate REPL'.  But I can see Tal's point, though the relative expansion is pretty small.  And there is nothing on the master completions issue #27609 where I collected known not-yet-rejected suggestions and ideas.

The implementation is trivial.  Add two new lines to autocomplete.py.  So you can easily patch a private copy.  I am preparing a minimal PR.

import keyword  # add
...
    def fetch_completions(self, what, mode):
...
                    bigl = eval("dir()", namespace)
                    bigl.extend(keyword.kwlist)  # add
                    bigl.sort()

True, False, and None are also in builtins, so cannot serve as a test.
---

A separate idea: annotate completion list, at least as an option, with 'keyword' or class, possibly prefixed with 'built-in', so 'built-in function', 'function', and so on.
msg349096 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-08-06 02:40
Thanks Terry, I used a similar patch. My main use case was around typing where normal shell autocompletes it and was curious if it was intentional. I didn't know that windows didn't give keywords. The keywords are short and added very rarely and perhaps the bigger completion list to actual usage might be low since no one opened this issue as Tal mentioned I am open to others feedback.
msg349362 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-10 22:04
After more research, I am more inclined to add the keywords to the global identifiers list. The IDLE doc says that they should be present.

"Show Completions
 Open a scrollable list allowing selection of keywords and attributes."

If this issue were rejected, the line above would have to be changed.  (Other needed updates are another issue.)  It was added to idlelib/help.txt (since replaced by help.html) by KBK in

Revision: 209de1f6ca1beaaa6b5eeb413f02e9c8c334ee50
Author: Kurt B. Kaiser <kbk@shore.net>
Date: 2/8/2007 5:58:18 PM
Message: ... Added AutoComplete instructions to IDLE Help.

(I checked 2.6 fetch_completions and keywords were not actually included in the global ('') list.)

The completion menu entry was copied to idle.rst in patches for #5066 by Jessica McKeller and Todd Rovito.  Both docs were updated.  Another 5 people, at least, reviewed, including 3 core developers.  It was committed by a 4th, Andrew Svetlov.  So 9 people besides me did not notice the discrepancy and may have thought 'keywords, fine', like I have been.

One can invoke Show Completions not following anything to get a list of names that one should not use.  For that use, keywords should be included.  This list currently has 230+ identifiers, 150+ all lowercase.  keyword.kwlist would add 32 (35 - 3 builtins).  It is not a huge increase.
msg349363 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-10 22:52
Since the word "main" is short and since the dunder prefix is common, I don't expect to get much value out of adding '__main__'.    ISTM, this just increases the risk of a false positive for a given dunder method.

> To be clear, I'm currently -1 on this suggestion.

I concur.
msg349365 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-10 23:24
Raymond, since there is no proposal to 'add __main__', I don't understand your response.  For the case in question, the completion list continues to be based on the keys in __main__.__builtins__.__dict__, and main__.__dict__, as it always has been.
msg373250 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-07 21:20
Cheryl said "This looks good" on the PR." while noting that True should not be added as  After trying out REPL completions in macOS Terminal, I *really* want to be able to type 'im'<tab> and have 'import' appear.  (When there is just one match, it is filled in without displaying a list of one item.)  I increasingly suffer from 'dystypia' (which I coined as the reverse of 'dyslexia'), and 'import' is one of my worst words.  And I have to type it daily.  On #17238, Ramchandra Apte also requested completion of 'import'.

Sorting keywords by length, we get:
>>> sorted(keyword.kwlist, key=lambda s: len(s))
['as', 'if', 'in', 'is', 'or', 'and', 'def', 'del', 'for', 'not', 'try', 'None', 'True', 'elif', 'else', 'from', 'pass', 'with', 'False', 'async', 'await', 'break', 'class', 'raise', 'while', 'yield', 'assert', 'except', 'global', 'import', 'lambda', 'return', 'finally', 'continue', 'nonlocal', '__peg_parser__']

I agree that adding 2 and 3 letter keywords is not useful. Among 4 letter keywords, None and True are already present from builtins.  'elif' and 'else' would need at least 3 and 4 keystrokes to complete ('e', <Tab>, Down for 'else', <Enter>).  'from' would need at least 4 because of 'filter' and 'frozenset'.  'pass' would need 3 because of 'pow'.  'with' would require at least 5 if 'while' were added.  So skip length 4 keywords also.

So I am changing the proposal to adding the 17 keywords (other than False, already present) of length 5 or more.  These include 'async' and 'await', requested by Karthikeyan in the opening post above.
msg373251 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2020-07-07 21:40
Auto-completion is not just about saving keystrokes. For example, I assume that many Python beginners just read through the completion list to see what the options are. This inconsistency would be hard to grok.

I think that including only some of the keywords in the completions list could potentially be very confusing. We'd have "class" but not "def", "finally" but not "else", "while" but not "for".

If the standard REPL completes keywords (at least on some platforms) that's a good enough argument to include them in IDLE, in my opinion.
msg373252 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2020-07-07 21:43
Also, note that the keywords would only be included in the suggested completions when not in a string and when not completing an attribute. So, for example, such a change could not possibly affect the completion of dunder method names.
msg373365 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-09 00:24
Tal, I suggested the compromise because of your original objection.  Since you think half is worse than all, I will revert the change.  It did get me to do a needed rewrite of the Completions section of the IDLE doc.
msg373429 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-09 22:08
New changeset bce2eb4646021910aa4074d86f44a09b32d0b2b2 by Terry Jan Reedy in branch 'master':
bpo-37765: Add keywords to IDLE tab completions (GH-15138)
https://github.com/python/cpython/commit/bce2eb4646021910aa4074d86f44a09b32d0b2b2
msg373430 - (view) Author: miss-islington (miss-islington) Date: 2020-07-09 22:54
New changeset fd27fb7f3dd157294f05bb060f7efd243732ab2d by Miss Islington (bot) in branch '3.9':
bpo-37765: Add keywords to IDLE tab completions (GH-15138)
https://github.com/python/cpython/commit/fd27fb7f3dd157294f05bb060f7efd243732ab2d
msg373431 - (view) Author: miss-islington (miss-islington) Date: 2020-07-09 22:54
New changeset 3d1c06e8b9eec5fc1ea2ed4dc1ea79c705da8ab8 by Miss Islington (bot) in branch '3.8':
bpo-37765: Add keywords to IDLE tab completions (GH-15138)
https://github.com/python/cpython/commit/3d1c06e8b9eec5fc1ea2ed4dc1ea79c705da8ab8
msg373432 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-09 23:47
PR 15138 always adds keywords to the big list for the current module.  They are also normally present in the small list, when it only excludes '_' names.  But if the module being edited contains '__all__', the small list, which is the first list presented, is currently just __all__.  This excludes builtins and now keywords and possibly non-_ names defined in the module.  I think this restriction is a mistake; __all__ defines a limited external view of the module.  It is not intended to restrict the names used in the module.  I will remove the restriction (and a crash bug it contains) in a partly completed PR for #37766.
History
Date User Action Args
2020-07-09 23:47:45terry.reedysetstatus: open -> closed
resolution: fixed
messages: + msg373432

stage: patch review -> resolved
2020-07-09 22:54:47miss-islingtonsetmessages: + msg373431
2020-07-09 22:54:21miss-islingtonsetmessages: + msg373430
2020-07-09 22:09:07miss-islingtonsetpull_requests: + pull_request20572
2020-07-09 22:09:00miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request20571
stage: test needed -> patch review
2020-07-09 22:08:40terry.reedysetmessages: + msg373429
2020-07-09 00:24:30terry.reedysetmessages: + msg373365
title: IDLE: Include longer keywords in __main__ autocomplete list -> IDLE: Include keywords in module-level autocomplete list
2020-07-07 21:43:10taleinatsetmessages: + msg373252
2020-07-07 21:40:02taleinatsetmessages: + msg373251
2020-07-07 21:21:13terry.reedysettitle: IDLE: Include 'long' keywords in __main__ autocomplete list -> IDLE: Include longer keywords in __main__ autocomplete list
2020-07-07 21:20:43terry.reedysettitle: IDLE: Include keywords in __main__ autocomplete list -> IDLE: Include 'long' keywords in __main__ autocomplete list
messages: + msg373250
versions: + Python 3.10, - Python 3.7
2019-08-10 23:24:12terry.reedysetmessages: + msg349365
2019-08-10 22:52:37rhettingersetmessages: + msg349363
2019-08-10 22:04:31terry.reedysetnosy: + cheryl.sabella

messages: + msg349362
stage: patch review -> test needed
2019-08-06 02:40:12xtreaksetnosy: + rhettinger
messages: + msg349096
2019-08-05 20:37:01terry.reedylinkissue27609 dependencies
2019-08-05 20:24:22terry.reedysetkeywords: + patch
stage: test needed -> patch review
pull_requests: + pull_request14876
2019-08-05 20:16:06terry.reedysetnosy: - rhettinger
title: Include keywords in autocomplete list for IDLE -> IDLE: Include keywords in __main__ autocomplete list
messages: + msg349067

versions: + Python 3.7, Python 3.8
stage: test needed
2019-08-05 18:30:19taleinatsetmessages: + msg349065
2019-08-05 18:29:29taleinatsetnosy: + rhettinger
messages: + msg349064
2019-08-05 18:03:48xtreaksetnosy: + taleinat
2019-08-05 18:03:28xtreakcreate