Author terry.reedy
Recipients peter.otten, rhettinger, terry.reedy
Date 2017-01-31.18:50:14
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1485888614.26.0.434548847959.issue29287@psf.upfronthosting.co.za>
In-reply-to
Content
This is 3 related but somewhat distinct proposals.

1. Special handling (normal syntax colorizing) of f-expressions (the grammatical term used at https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals).  

2. Brace matching within the strings.  (For normal strings, this would be nice when the string is to be used as a format string, but there is no way to know how the string will be used.)

3. Normal identifier autocompletion (as opposed to within-string file name autocompletion).

I have two types of doubts about #1

1. Should it be done?
 a. I have not yet used f-strings, so I not know if I would want uncolorized holes within them.
 b. Do beginners use f-strings?  Would they find this more useful than confusing?
 c. Is this the sort of 'advanced feature' that Guido has said should *not* be copied from other editors?

2. Can it be done sensibly within the limits of IDLE's colorizer. 

IDLE's colorizer defines a giant regex that joins regexes that match keywords, builtins, comments, and strings (and newlines for synchronization).  Each of the latter is a named capturing group that joins alternatives for that group.  Keywords and built-in names are recognized when complete.  Partial comments and strings are recognized as soon as '#' or an open quote is typed.  There is a human-verified test of colorizing that could, I believe, be turned into a unit test of the re matching.  This would be needed for approach b. below.

The compiled re is used in a ColorDelegator instance that is part of a chain of delegators tied to a text widget.  The class code is not documented and I do not understand it well enough to modify it without adding tests.  But it was not designed for easy testing.

Sidenote: There are DEBUG prints in multiple methods (but not inrecolorize_main).  Some messages can come from multiple methods.  I should add message to the r...main method and prefix all messages with an indicator of the source so the control flow is easier to follow

I see two possible approaches to separately colorizing f-expressions within an f-string.

a. Follow the example of 'def' and 'class'.  They are recognized as a special case (of builting) and when they occur, a separate 'if' clause and re is used to colorize the following name.

The problem with doing this with f-strings is that we want to recursively apply the re...main function to a short substring, and the function is not designed for that.  We also want to do this separately for each embedded f-expression.  It might work to write a reduced version of recolorize_main as recolorize_fexp.

This approach would allow for {} matching once a closing quote is typed, but not identifier autocompletion.

b. Do the special-casing by writing special regexes to recognize a null f-string (no embedded f-expression), and beginning, middle, and ending string parts of an f-string.  But I don't know if it is possible to write an re that will *only* match null f-strings.

That aside, the f-expression would then be treated normally, and autocomplete should just work.  {} matching would be harder.  Without adding new state variables, I imagine that the end quote of the invalid f"a{b" would be seen as the beginning of a new string.
History
Date User Action Args
2017-01-31 18:50:14terry.reedysetrecipients: + terry.reedy, rhettinger, peter.otten
2017-01-31 18:50:14terry.reedysetmessageid: <1485888614.26.0.434548847959.issue29287@psf.upfronthosting.co.za>
2017-01-31 18:50:14terry.reedylinkissue29287 messages
2017-01-31 18:50:14terry.reedycreate