➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ast and tokenize disagree about line number
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: CCLDArjun, Mark.Shannon, nitishch
Priority: normal Keywords:

Created on 2018-01-12 12:16 by Mark.Shannon, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
tokenize_fail_test.py Mark.Shannon, 2018-01-12 12:16
Messages (4)
msg309852 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2018-01-12 12:16 
This occurs (on linux at least) when the the end of line sequence `\r\r\n` occurs in a comment that is indented relative to the following lines.

The attached file demonstrates the problem.
msg310155 - (view) Author: Nitish (nitishch) * Date: 2018-01-17 11:50 
What should happen ideally? The stray '\r' be treated like a whitespace?
msg396113 - (view) Author: Arjun (CCLDArjun) * Date: 2021-06-19 02:09 
the line numbers are different for me too (MacOS).

it seems that tokenize module is off by +1 line
msg396116 - (view) Author: Arjun (CCLDArjun) * Date: 2021-06-19 04:52 
Well actually, it depends on the platform?
History
Date User Action Args
2022-04-11 14:58:56adminsetgithub: 76717
2021-06-19 04:52:06CCLDArjunsetmessages: + msg396116
2021-06-19 02:09:51CCLDArjunsetnosy: + CCLDArjun
messages: + msg396113
2021-06-18 15:03:31iritkatrielsetversions: + Python 3.11, - Python 2.7
2018-01-17 11:50:32nitishchsetmessages: + msg310155
2018-01-16 16:52:09nitishchsetnosy: + nitishch
2018-01-12 12:16:49Mark.Shannoncreate