Message103799
When altering tokens and thus not providing token location information, tokenize.untokenize sometimes misses out the first token. Failure case below.
Expected output: 'import foo ,bar\n'
Actual output: 'foo ,bar\n'
$ python
Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import StringIO, tokenize
>>>
>>> def strip(iterable):
... for t_type, t_str, (srow, scol), (erow, ecol), line in iterable:
... yield t_type, t_str
...
>>> source = StringIO.StringIO('import foo, bar\n')
>>> print repr(tokenize.untokenize(strip(tokenize.generate_tokens(source.readline))))
'foo ,bar \n'
>>> source.seek(0)
>>> print repr(tokenize.untokenize(tokenize.generate_tokens(source.readline)))
'import foo, bar\n'
>>> |
|
Date |
User |
Action |
Args |
2010-04-21 02:18:32 | rb | set | recipients:
+ rb |
2010-04-21 02:18:32 | rb | set | messageid: <1271816312.19.0.485614247328.issue8478@psf.upfronthosting.co.za> |
2010-04-21 02:18:30 | rb | link | issue8478 messages |
2010-04-21 02:18:29 | rb | create | |
|