Author gumblex
Recipients gumblex
Date 2015-06-14.03:33:12
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1434252794.24.0.0112893923697.issue24447@psf.upfronthosting.co.za>
In-reply-to
Content
If a script uses tabs for indentation, tokenize.untokenize won't restore original indentation correctly from the second line of the indentation level, and thus breaks the file.

This affects all Python versions.

Test code:

python2 -c 'import sys, tokenize; sys.stdout.write(tokenize.untokenize(tokenize.generate_tokens(sys.stdin.readline)))' < tab.py

python3 -c 'import sys, tokenize; sys.stdout.buffer.write(tokenize.untokenize(tokenize.tokenize(sys.stdin.buffer.readline)))' < tab.py

Out:

def foo():
	pass
 pass
 if 1:
		pass
  pass
History
Date User Action Args
2015-06-14 03:33:14gumblexsetrecipients: + gumblex
2015-06-14 03:33:14gumblexsetmessageid: <1434252794.24.0.0112893923697.issue24447@psf.upfronthosting.co.za>
2015-06-14 03:33:14gumblexlinkissue24447 messages
2015-06-14 03:33:13gumblexcreate