This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author noam
Recipients georg.brandl, noam
Date 2008-06-02.11:33:19
SpamBayes Score 0.039297316
Marked as misclassified No
Message-id <1212406411.84.0.811061181289.issue3028@psf.upfronthosting.co.za>
In-reply-to
Content
Hello,

The documentation of the tokenize module says: "The line passed is the
*logical* line; continuation lines are included."

Some background: The tokenize module splits a python source into tokens,
and says for each token where it begins and where it ends, in the format
of (row, offset). This note in the documentation made me think that
continuation lines are considered as one line, and made me break my head
how I should find the offset of the token in the original string. The
truth is that the row number is simply the index of the line as returned
by the readline function, and it's very simple to reconstruct the string
offset.

I suggest that this will be changed to something like "The line passed
is the index of the string returned by the readline function, plus 1.
That is, the first string returned is called line 1, the second is
called line 2, and so on."

Thanks,
Noam
History
Date User Action Args
2008-06-02 11:33:32noamsetspambayes_score: 0.0392973 -> 0.039297316
recipients: + noam, georg.brandl
2008-06-02 11:33:32noamsetspambayes_score: 0.0392973 -> 0.0392973
messageid: <1212406411.84.0.811061181289.issue3028@psf.upfronthosting.co.za>
2008-06-02 11:33:25noamlinkissue3028 messages
2008-06-02 11:33:23noamcreate