Author miedzinski
Recipients miedzinski
Date 2019-07-18.14:13:50
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
Both C and Python tokenizers emit NEWLINE tokens on continued blank lines. Because of this it is possible to emit repeated NEWLINEs, which some tools don't expect to happen.

I've attached example source file which is tokenized into 3 NEWLINE tokens in a row.
Date User Action Args
2019-07-18 14:13:50miedzinskisetrecipients: + miedzinski
2019-07-18 14:13:50miedzinskisetmessageid: <>
2019-07-18 14:13:50miedzinskilinkissue37621 messages
2019-07-18 14:13:50miedzinskicreate