Message209964
I read the manual more carefully and noticed that the guarantee is that tokenizing the result of untokenize matches the input to untokenize. " The result is guaranteed to tokenize back to match the input so that the conversion is lossless and round-trips are assured."
I ran the test in 3.4 and noticed the exactness and direction of the matching does not matter because only the 2nd \t is replaced by a space, making the invalid code that raises IndentationError.
So this is definitely a bug and I withdraw the close suggestion.
I believe the test should be that both lines of the body begin with the same indent.
from tokenize import tokenize, untokenize
import io
def test_tab_indent(self):
code = b"if False:\n\tx=3\n\ty=3\n"
codelines = untokenize(tokenize(io.BytesIO(s).readline)).split(\n)
assertEqual(codelines[1], codelines[2])
Of course, the test of tokenize and untokenize should be separated. |
|
Date |
User |
Action |
Args |
2014-02-02 08:30:17 | terry.reedy | set | recipients:
+ terry.reedy, jaraco, Arfrever |
2014-02-02 08:30:17 | terry.reedy | set | messageid: <1391329817.53.0.82097480776.issue20387@psf.upfronthosting.co.za> |
2014-02-02 08:30:17 | terry.reedy | link | issue20387 messages |
2014-02-02 08:30:17 | terry.reedy | create | |
|