Title: Teach IDLE to Autocomplete dictionary keys
Type: enhancement Stage: patch review
Components: IDLE Versions: Python 3.7, Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Eduardo.Seabra, cdspace, louielu, martin.panter, rhettinger, terry.reedy
Priority: normal Keywords: easy, patch

Created on 2014-04-16 18:21 by rhettinger, last changed 2017-05-12 01:58 by terry.reedy.

File name Uploaded Description Edit
issue21261.patch Eduardo.Seabra, 2014-06-20 00:34 review
issue21261.patch Eduardo.Seabra, 2014-06-20 17:53 With tests review
Pull Requests
URL Status Linked Edit
PR 1511 closed louielu, 2017-05-09 06:08
Messages (12)
msg216542 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-04-16 18:21
IDLE can autocomplete global variable, method names, and filenames.  But, it cannot complete dictionary keys.

Here's what we want:

   >>> d = {'long_key': '10, 'short_key': 20}
   >>> d['lo<tab>
msg216813 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-04-18 22:09
Looks sensible.
msg221056 - (view) Author: Eduardo Seabra (Eduardo.Seabra) * Date: 2014-06-20 00:34
From the example, I couldn't know if the patch should also autocomplete int and other types. So here's a patch that autocompletes string dictionary keys.
I'm new contributing so let me know if I made anything wrong and I'll fix as soon as possible.
msg221061 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-06-20 02:08
String keys is what Raymond requested and what looks sensible to me. A week ago, I committed test_idle/ and an incomplete So we can now, for this issue, at least partly follow our standard procedure of requiring tests with patches.

I want at least a test of is_in_subscript_string(self) added to test_hyperparser. cls.code will need another line (or an existing string bracketed and the string test altered) and a new test_is_in_subscript_string added. If you are working from an installation rather that a repository, and therefore cannot write/test an addition to the new file, say so.

The change to auto-complete is trickier. I would have to look the code versus to tests to decide what to do, if anything. I might decide to improve the autocomplete htest (human test, in repository and test_idle/ rather than the unittest.

In any case, we need a signed contributor agreement to accept patches.
msg221102 - (view) Author: Eduardo Seabra (Eduardo.Seabra) * Date: 2014-06-20 17:53
I've added three lines to cls.code to test_hyperparser. So I can test for subscripts with double quotes, single quotes and with no strings at all.

Should I implement try_open_completions_event for COMPLETE_DICTIONARY? Calling this event everytime someone types a string seemed a bit expensive in my opinion.

I'm attaching the new patch.

As fas as the signed contributor, I've already signed last week but still waiting.
msg265475 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-05-13 13:29
I have no idea how IDLE works internally, but I wonder if it is possible to share some of the work with the Readline completer (rlcompleter module). Despite the name, rlcompleter should be usable without also using Readline, though recently (Issue 25660) I think this was broken. Anyway, maybe see Issue 10351 for the beginning of an rlcompleter patch and some potential test cases.
msg265489 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-05-13 21:12
Thanks.  Being on Windows, I never paid attention to rlcompleter.  The big difference is that IDLE completions use IDLE's hyperparser module (which is used for other purposes also).  I will look at the tests.
msg293282 - (view) Author: Louie Lu (louielu) * Date: 2017-05-09 06:11
In this PR, it will complete dictionary key with string, int, and others.

for example:

    d = {'long_key': 10, 'short_key': 20, 30: 40, (((1, 2), 3, 4), 5): 50}
    d['lo<tab>  -> d['long_key'
    d[(((1<tab> -> d[(((1, 2), 3, 4), 5)
    d[3<tab>    -> d[30

The problem is, autocomplete_w can't figure the original key is string or others, so this will be possible:

    d[long<tab> -> d[long_key]
    d[shor<tab> -> d[short_key]
msg293284 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-05-09 06:38
Would it be safer/simpler to just autocomplete string keys.
msg293289 - (view) Author: Louie Lu (louielu) * Date: 2017-05-09 07:59
I'm not sure the "safer" meaning. If it is about for beginner less confuse when mistakenly typing "d[long_<tab>", the answer will be yes for only complete string keys.

Impl complexity between str-only and not-str-only will not have too much different, only need to change the sentinel and some other work.
msg293502 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-05-11 16:03
I think it would be unusual for tab completion to work on non-strings and would create a weird feel to the API.
msg293520 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-12 01:58
IDLE currently completes global names, attributes after ., and filename segments after / or \ within a string.  In the later two cases, a box will pop up automatically after a user selected time after typing . or /\ and nothing thereafter.  The filename segments are not quoted in the list box.

These completions work within subscripts.
d[a<tab or wait> pops up global name completion box
d['/<tab or wait> pops up filename completion box

Raymond proposes that IDLE complete 'dictionary [string] keys'.  To properly code and test, we need a more complete specification.  For instance, "a string key box should open after an opening quote that follows '[' that follows a dict expression".  Any opening quote should work, just as for filename completion.

This is similar "a calltip opens after a '(' that follows a callable expresssion".  For calltips, the expression cannot contain a function call, because calls can take an indeterminant amount of time.  If "expression.find('(') != -1", the calltip is aborted and the same should be true here.  Also, calltips.get_entity(expression) should be reused to get the dict object.  (test_calltips should but does not test that 'f()(' is ignored and get_entity not called.  The same should be true for "f()['".)

Nice (?) but not necessary: delayed auto-popup after typing "d[<open quote>".  This seems that it would be more difficult than the current auto popups.  And see the following.

This proposal conflicts with filename completion for subscripts.  When one is accessing an existing value, one would want key completion.  If one is assigning a value to a new filename key, one would want filename completion.  The simplest solution I can think of is to not auto pop up key completion but to require <tab> before typing (/\) and waiting.

Lastly, should the string keys be quoted in the box?
| long key  |
| short key |
|'long key' |
|'short key'|

Selecting key objects by their representation is tempting, but it is conceptually different from completing names.  Objects may have one canonical representation, but many possible representations.  So clicking on a list  (which currently does not work) or using movement keys is more sensible than typing chars that have to match one of many possibilities.  String keys would have to be quoted.

So I would only consider this as a separate issue, depending on a fix for clicks.  It should only be accessed by <tab> immediately after '[', and I might want to disable selection by character matching.

Even then, I would be dubious.  I grepped idlelib for "\w\[".  A majority of subscripts are names, handled by current name completion or not (if the names are local, which they often are).  The rest are either list indexes and slices involving literal ints or string keys, which this proposal would handle for accessible dicts.  I am pretty sure there are no keys other than names and strings.

But the sparsity of use cases is my problem even with this proposal. Calltips are useful because there are many globally accessible callables, including builtins and imports.  But other than class __dicts__, there are few globally accessible dicts, except perhaps in toy beginner code.  Raymond, have I missed something?

The idlelib grep had 763 hits and I believe more that half are for dicts.  But they may all be locals or self attributes.  I would love to be able, for instance, to type "local_dict['<tab>" and fill in 'background', but that will not work.
Date User Action Args
2017-05-12 01:58:55terry.reedysetversions: - Python 2.7, Python 3.5
2017-05-12 01:58:37terry.reedysetstage: test needed -> patch review
messages: + msg293520
versions: + Python 3.6, Python 3.7, - Python 3.4
2017-05-11 16:03:28rhettingersetmessages: + msg293502
2017-05-09 07:59:14louielusetmessages: + msg293289
2017-05-09 06:38:11rhettingersetmessages: + msg293284
2017-05-09 06:11:20louielusetnosy: + louielu
messages: + msg293282
2017-05-09 06:08:04louielusetpull_requests: + pull_request1612
2016-07-25 00:08:07terry.reedylinkissue27609 dependencies
2016-05-13 21:12:44terry.reedysetmessages: + msg265489
2016-05-13 13:29:49martin.pantersetnosy: + martin.panter
messages: + msg265475
2016-05-13 12:03:26terry.reedylinkissue27013 superseder
2014-06-20 17:53:06Eduardo.Seabrasetfiles: + issue21261.patch

messages: + msg221102
2014-06-20 02:08:35terry.reedysetmessages: + msg221061
stage: test needed
2014-06-20 00:34:28Eduardo.Seabrasetfiles: + issue21261.patch

nosy: + Eduardo.Seabra
messages: + msg221056

keywords: + patch
2014-05-22 22:11:49cdspacesetnosy: + cdspace
2014-04-18 22:09:53terry.reedysetnosy: + terry.reedy
messages: + msg216813
2014-04-16 20:03:59rhettingersetkeywords: + easy
2014-04-16 18:21:12rhettingercreate