Message219614
By the docs,
Except at the beginning of a logical line or in
string literals, the whitespace characters space,
tab and formfeed can be used interchangeably to
separate tokens. Whitespace is needed between two
tokens only if their concatenation could otherwise
be interpreted as a different token
(e.g., ab is one token, but a b is two tokens).
"_ if 1else _" should compile equivalently to "_ if 1 else _".
The tokenize module does this correctly:
import io
import tokenize
def print_tokens(string):
tokens = tokenize.tokenize(io.BytesIO(string.encode("utf8")).readline)
for token in tokens:
print("{:12}{}".format(tokenize.tok_name[token.type], token.string))
print_tokens("_ if 1else _")
#>>> ENCODING utf-8
#>>> NAME _
#>>> NAME if
#>>> NUMBER 1
#>>> NAME else
#>>> NAME _
#>>> ENDMARKER
but it fails when compiled with, say, "compile", "eval" or "ast.parse".
import ast
compile("_ if 1else _", "", "eval")
#>>> Traceback (most recent call last):
#>>> File "", line 32, in <module>
#>>> File "<string>", line 1
#>>> _ if 1else _
#>>> ^
#>>> SyntaxError: invalid token
eval("_ if 1else _")
#>>> Traceback (most recent call last):
#>>> File "", line 40, in <module>
#>>> File "<string>", line 1
#>>> _ if 1else _
#>>> ^
#>>> SyntaxError: invalid token
ast.parse("_ if 1else _")
#>>> Traceback (most recent call last):
#>>> File "", line 48, in <module>
#>>> File "/usr/lib/python3.4/ast.py", line 35, in parse
#>>> return compile(source, filename, mode, PyCF_ONLY_AST)
#>>> File "<unknown>", line 1
#>>> _ if 1else _
#>>> ^
#>>> SyntaxError: invalid token
Further, some other forms work:
1 if 0b1else 0
#>>> 1
1 if 1jelse 0
#>>> 1
See
http://stackoverflow.com/questions/23998026/why-isnt-this-a-syntax-error-in-python
particularly,
http://stackoverflow.com/a/23998128/1763356
for details. |
|
Date |
User |
Action |
Args |
2014-06-02 18:24:00 | Joshua.Landau | set | recipients:
+ Joshua.Landau |
2014-06-02 18:24:00 | Joshua.Landau | set | messageid: <1401733440.77.0.120432390188.issue21642@psf.upfronthosting.co.za> |
2014-06-02 18:24:00 | Joshua.Landau | link | issue21642 messages |
2014-06-02 18:24:00 | Joshua.Landau | create | |
|