➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author serhiy.storchaka
Recipients ammar2, gregory.p.smith, meador.inge, pablogsal, serhiy.storchaka, taleinat, terry.reedy
Date 2018-10-30.06:45:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1540881957.75.0.788709270274.issue35107@psf.upfronthosting.co.za>
In-reply-to
Content 
I am surprised, that removing the newline character adds a token:

>>> pprint.pprint(list(tokenize.generate_tokens(io.StringIO('#\n').readline)))
[TokenInfo(type=55 (COMMENT), string='#', start=(1, 0), end=(1, 1), line='#\n'),
 TokenInfo(type=56 (NL), string='\n', start=(1, 1), end=(1, 2), line='#\n'),
 TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]
>>> pprint.pprint(list(tokenize.generate_tokens(io.StringIO('#').readline)))
[TokenInfo(type=55 (COMMENT), string='#', start=(1, 0), end=(1, 1), line='#'),
 TokenInfo(type=56 (NL), string='', start=(1, 1), end=(1, 1), line='#'),
 TokenInfo(type=4 (NEWLINE), string='', start=(1, 1), end=(1, 2), line=''),
 TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]
History
Date User Action Args
2018-10-30 06:45:57serhiy.storchakasetrecipients: + serhiy.storchaka, terry.reedy, gregory.p.smith, taleinat, meador.inge, ammar2, pablogsal
2018-10-30 06:45:57serhiy.storchakasetmessageid: <1540881957.75.0.788709270274.issue35107@psf.upfronthosting.co.za>
2018-10-30 06:45:57serhiy.storchakalinkissue35107 messages
2018-10-30 06:45:57serhiy.storchakacreate