This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author mallyvai
Recipients mallyvai
Date 2009-04-27.20:26:52
SpamBayes Score 6.8628436e-13
Marked as misclassified No
Message-id <1240864016.41.0.600109014145.issue5857@psf.upfronthosting.co.za>
In-reply-to
Content
Returning an anonymous 5-tuple seems like a suboptimal interface since
it's so easy to accidentally confuse, for example, the indices of start
and end. I've used tokenize.py for several scripts in the past few weeks
and I've always ended up writing some sort of wrapper function for
generate_tokens that names the returned tuple's fields to help me avoid
mistakes like this.

I'd like to propose the following patch that simply decorates the
generate_token function and names its return values' fields. Since it's
a namedtuple, it should be fully backwards compatible with the existing
interface, but also allow member access via 

next_token.type
next_token.string
next_token.start.row, next_token.start.col
next_token.end.row, next_token.end.col
next_token.line

If this seems like a reasonable way to do things, I'd be happy to submit
relevant doc patches as well as the corresponding patch for 3.0.
History
Date User Action Args
2009-04-27 20:26:56mallyvaisetrecipients: + mallyvai
2009-04-27 20:26:56mallyvaisetmessageid: <1240864016.41.0.600109014145.issue5857@psf.upfronthosting.co.za>
2009-04-27 20:26:54mallyvailinkissue5857 messages
2009-04-27 20:26:53mallyvaicreate