classification
Title: Accelerate 'string'.format(value, ...) by using formatted string literals
Type: performance Stage:
Components: Interpreter Core Versions: Python 3.7
process
Status: open Resolution:
Dependencies: 11549 Superseder:
Assigned To: serhiy.storchaka Nosy List: eric.smith, serhiy.storchaka
Priority: low Keywords:

Created on 2016-09-29 08:50 by serhiy.storchaka, last changed 2017-12-25 16:57 by serhiy.storchaka.

Messages (2)
msg277689 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-29 08:50
For now using formatted string literals (PEP498) is the fastest way of formatting strings.

$ ./python -m perf timeit -s 'k = "foo"; v = "bar"' -- '"{!s} = {!r}".format(k, v)'
Median +- std dev: 3.96 us +- 0.17 us

$ ./python -m perf timeit -s 'k = "foo"; v = "bar"' -- 'f"{k!s} = {v!r}"'
Median +- std dev: 1.09 us +- 0.08 us

The compiler could translate new-style formatting with literal format string to the equivalent formatted string literal. The code '{!s} = {!r}'.format(k, v) could be translated to

    t1 = k; t2 = v; f'{t1!r} = {t2!s}'; del t1, t2

or even simpler if k and v are initialized local variables.

$ ./python -m perf timeit -s 'k = "foo"; v = "bar"' -- 't1 = k; t2 = v; f"{t1!s} = {t2!r}"; del t1, t2'
Median +- std dev: 1.22 us +- 0.05 us

This is not easy issue and needs first implementing the AST optimizer.
msg277693 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-09-29 09:09
One thing to be careful of here is that there's one slight difference between how str.format() and f-strings handle indexing of values. f-strings, of course, use normal Python semantics, but 
str.format() treats indexing by things that don't look like integers as string literals, not variables. It's an unfortunate left-over from the original PEP-3101 specification:

>>> d = {'a':'string', 0:'integer'}
>>> a = 0
>>> f'{d[0]}'
'integer'
>>> '{d[0]}'.format(d=d)
'integer'
>>> f'{d[a]}'
'integer'
>>> '{d[a]}'.format(d=d)
'string'

Note that the exact same expression {d[a]} is evaluated differently by the two ways to format.

There's a test for this in test_fstring.py.

Someday, I'd like to deprecate this syntax in str.format(). I don't think it could ever be added back in, because it requires either additional named parameters which aren't used as formatting parameters, or it requires global/local lookups (which isn't going to happen).

i.e., this:
'{d[a]}'.format(d=d, a=a)
History
Date User Action Args
2017-12-25 16:57:46serhiy.storchakasetassignee: serhiy.storchaka
2016-09-29 09:09:55eric.smithsetmessages: + msg277693
2016-09-29 08:50:26serhiy.storchakasetdependencies: + Build-out an AST optimizer, moving some functionality out of the peephole optimizer
2016-09-29 08:50:04serhiy.storchakacreate