Issue32954
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2018-02-26 10:19 by arcivanov, last changed 2022-04-11 14:58 by admin.
Messages (9) | |||
---|---|---|---|
msg312909 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:19 | |
I'd like to start a discussion on/gauge interest for introducing an enhancement to PEP-498 in a form of delayed/lazy/lambda f-string. The proposed change is, conceptually, minor and would not differ from PEP-498 syntactically at all except for string prefix. E.g. extra = f'{extra},waiters:{len(self._waiters)}' becomes extra = fl'{extra},waiters:{len(self._waiters)}' The proposal would produce a lambda-like object x('format'), where x.__str__() == f'format'. This should come extremely useful in all cases where delayed or conditional string formatting and concatenation are desired, such as in cases of logging. As an example, currently, the logger.debug("Format %s string", value) cannot be used with an f-string as follows logger.debug(f"Format {value} string") without an unconditional evaluation of all parameters due to current compilation prior to logging checking whether it's even enabled for debug: >>> b = 1 >>> def a(x): ... return f"Foo {x} bar {b}" ... >>> dis.dis(a) 2 0 LOAD_CONST 1 ('Foo ') 2 LOAD_FAST 0 (x) 4 FORMAT_VALUE 0 6 LOAD_CONST 2 (' bar ') 8 LOAD_GLOBAL 0 (b) 10 FORMAT_VALUE 0 12 BUILD_STRING 4 14 RETURN_VALUE Additional great optimizations may be rendered by introducing an a fl-string is a case where foo = "foo" s1 = fl"S1 value ${foo}" s2 = fl"S2 value of ${s1}" print(s2) may produce only one BUILD_STRING instruction, potentially dramatically increasing performance in heavy string-concat based applications. Even when a compiler will not be able to statically prove that a particular value is in fact an fl-string, an interpreter-level check or a JIT-based Python implementation may be able to optimize such concats ad-nauseam (with trap-based deopt), allowing to collapse an extremely long chains of formats into a single BUILD_STRING op without any intermediary string allocation/concatenation (very useful in cases such as web servers, templating engines etc). I'll be happy to produce patches against 3.7/3.8 for this if a general concept is found useful/desirable by the community. |
|||
msg312910 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2018-02-26 10:27 | |
class FL: def __init__(self, func): self.func = func def __str__(self): return self.func() extra = FL(lambda: f'{extra},waiters:{len(self._waiters)}') |
|||
msg312911 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:42 | |
As an example this is the current state of affairs: >>> def x(): ... foo = "foo" ... s1 = f"S1 value {foo}" ... s2 = f"S2 value {s1}" ... print(s2) ... >>> dis.dis(x) 2 0 LOAD_CONST 1 ('foo') 2 STORE_FAST 0 (foo) 3 4 LOAD_CONST 2 ('S1 value ') 6 LOAD_FAST 0 (foo) 8 FORMAT_VALUE 0 10 BUILD_STRING 2 12 STORE_FAST 1 (s1) 4 14 LOAD_CONST 3 ('S2 value ') 16 LOAD_FAST 1 (s1) 18 FORMAT_VALUE 0 20 BUILD_STRING 2 22 STORE_FAST 2 (s2) 5 24 LOAD_GLOBAL 0 (print) 26 LOAD_FAST 2 (s2) 28 CALL_FUNCTION 1 30 POP_TOP 32 LOAD_CONST 0 (None) 34 RETURN_VALUE whereas an fl-string representation of: >>> def x(): ... foo = "foo" ... s1 = fl"S1 value {foo}" ... s2 = fl"S2 value {s1}" ... print(s2) ... would behave close to: >>> def x(): ... foo = "foo" ... s1 = lambda: f"S1 value {foo}" ... s2 = lambda: f"S2 value {s1()}" ... print(s2()) ... >>> dis.dis(x) 2 0 LOAD_CONST 1 ('foo') 2 STORE_DEREF 0 (foo) 3 4 LOAD_CLOSURE 0 (foo) 6 BUILD_TUPLE 1 8 LOAD_CONST 2 (<code object <lambda> at 0x7ff2ef1abd20, file "<stdin>", line 3>) 10 LOAD_CONST 3 ('x.<locals>.<lambda>') 12 MAKE_FUNCTION 8 14 STORE_DEREF 1 (s1) 4 16 LOAD_CLOSURE 1 (s1) 18 BUILD_TUPLE 1 20 LOAD_CONST 4 (<code object <lambda> at 0x7ff2ef1abae0, file "<stdin>", line 4>) 22 LOAD_CONST 3 ('x.<locals>.<lambda>') 24 MAKE_FUNCTION 8 26 STORE_FAST 0 (s2) 5 28 LOAD_GLOBAL 0 (print) 30 LOAD_FAST 0 (s2) 32 CALL_FUNCTION 0 34 CALL_FUNCTION 1 36 POP_TOP 38 LOAD_CONST 0 (None) 40 RETURN_VALUE Where LOAD_CONST, LOAD_CONST, MAKE_FUNCTION sequences are replaced with BUILD_LAZY_STRING (name's provisional) producing the same lambda-like formatter function that cooperates with FORMAT_VALUE and BUILD_STRING/BUILD_LAZY_STRING ops. |
|||
msg312912 - (view) | Author: Eric V. Smith (eric.smith) * | Date: 2018-02-26 10:51 | |
See also PEP 501. |
|||
msg312913 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:54 | |
@serhiy.storchaka Of course a similar pattern can be implemented via a class (or even without one as I've shown below). But you can clearly notice that in your example: 1) There are tons of boilerplate (as in mine with lambdas). 2) It's going to be slower than a core implementation with specialized objects. 3) It can't be made to cooperate with a FORMAT_VALUE/BUILD_STRING - intermediate strings will still be produced. |
|||
msg312914 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:59 | |
@eric.smith Thanks! I was looking for such a general-purpose proposal but could not find it. Although general-purpose mechanism that would allow pluggable constructs like `sh`, `html`, `sql` and the like is awesome and very desirable (especially sh in preference of Popen madness), string formatting/concatenation is IMO a fundamental concept (hence PEP-498 is in core with specialized opcodes instead of being deferred to be a general purpose pluggable PEP-501 implementation). |
|||
msg312918 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 11:24 | |
> Although general-purpose mechanism that would allow pluggable constructs like `sh`, `html`, `sql` Strike that one, I didn't read into PEP-0501 deep enough before replying. Yes, `i"format"` is what I'm talking about. |
|||
msg312922 - (view) | Author: Eric V. Smith (eric.smith) * | Date: 2018-02-26 12:20 | |
Arcadiy: Somehow you're dropping Serhiy and me from the nosy list. I've re-added us. |
|||
msg312923 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 12:46 | |
Sorry about that! I'll be sure to refresh next time before posting a reply. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:58 | admin | set | github: 77135 |
2019-12-18 06:25:31 | mbdevpl | set | nosy:
+ mbdevpl |
2018-02-26 16:10:28 | barry | set | nosy:
+ barry |
2018-02-26 12:46:52 | arcivanov | set | messages: + msg312923 |
2018-02-26 12:20:52 | eric.smith | set | nosy:
+ eric.smith, serhiy.storchaka messages: + msg312922 |
2018-02-26 11:24:30 | arcivanov | set | messages: + msg312918 |
2018-02-26 10:59:46 | arcivanov | set | messages: + msg312914 |
2018-02-26 10:54:18 | arcivanov | set | nosy:
- eric.smith, serhiy.storchaka messages: + msg312913 |
2018-02-26 10:51:24 | eric.smith | set | nosy:
+ eric.smith, serhiy.storchaka messages: + msg312912 |
2018-02-26 10:42:52 | arcivanov | set | nosy:
- serhiy.storchaka messages: + msg312911 components: + Interpreter Core, - Library (Lib) |
2018-02-26 10:27:49 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg312910 |
2018-02-26 10:19:19 | arcivanov | create |