classification
Title: IDLE - fatal error when opening a file with certain tokenizing errors
Type: behavior Stage: resolved
Components: IDLE Versions: Python 3.4, Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: kbk, python-dev, roger.serwy, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2012-11-18 17:37 by roger.serwy, last changed 2012-12-27 20:11 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
editor_token_error.patch roger.serwy, 2012-11-18 17:37 review
sample_token_error.py roger.serwy, 2012-11-18 17:39
editor_token_error_rev1.patch roger.serwy, 2012-11-18 19:06 review
Messages (9)
msg175895 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-11-18 17:37
IDLE's IndentSearcher class in EditorWindow.py can raise an uncaught IndentationError when opening a file. The attached patch fixes the problem by catching everything that the tokenize module can raise explicitly, namely IndentationError, TokenError, and SyntaxError.

Alternatively, the except clause can be left bare, as it is disappointing to crash IDLE when simply guessing the indent width of a file. (See "guess_indent" in EditorWindow.py)
msg175896 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-11-18 17:39
I encountered this behavior while testing an extension that highlights tabs and trailing whitespace. The very first test file I wrote caused this crash. See the attached "sample_token_error.py".
msg175901 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-18 18:38
IndentationError is a subclass of SyntaxError.
msg175904 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-11-18 19:06
That's a good point. Attached is a revision to omit IndentationError.

Thanks Serhiy.
msg176266 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-11-24 03:08
Serhiy, was msg176255 meant for issue16491?
msg176268 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-24 08:11
> Serhiy, was msg176255 meant for issue16491?

Yes, sorry.
msg176890 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-04 10:17
LGTM.
msg178313 - (view) Author: Roundup Robot (python-dev) Date: 2012-12-27 20:05
New changeset d3c81ef728ae by Serhiy Storchaka in branch '2.7':
Issue #16504: Catch SyntaxErrors raised by tokenizer in IDLE.
http://hg.python.org/cpython/rev/d3c81ef728ae

New changeset e1ef04cfb57c by Serhiy Storchaka in branch '3.2':
Issue #16504: Catch SyntaxErrors raised by tokenizer in IDLE.
http://hg.python.org/cpython/rev/e1ef04cfb57c

New changeset f111e7078d54 by Serhiy Storchaka in branch '3.3':
Issue #16504: Catch SyntaxErrors raised by tokenizer in IDLE.
http://hg.python.org/cpython/rev/f111e7078d54

New changeset 69a2f6a7c257 by Serhiy Storchaka in branch 'default':
Issue #16504: Catch SyntaxErrors raised by tokenizer in IDLE.
http://hg.python.org/cpython/rev/69a2f6a7c257
msg178316 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-27 20:11
Thank you, Roger. Fixed.
History
Date User Action Args
2012-12-27 20:11:01serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg178316

stage: commit review -> resolved
2012-12-27 20:05:11python-devsetnosy: + python-dev
messages: + msg178313
2012-12-27 19:16:57serhiy.storchakasetassignee: serhiy.storchaka
2012-12-04 10:17:20serhiy.storchakasetnosy: + kbk

messages: + msg176890
stage: commit review
2012-11-24 08:12:04serhiy.storchakasetmessages: - msg176255
2012-11-24 08:11:39serhiy.storchakasetmessages: + msg176268
2012-11-24 03:08:45roger.serwysetmessages: + msg176266
2012-11-23 21:37:04serhiy.storchakasetmessages: + msg176255
2012-11-23 20:09:53terry.reedysetnosy: + terry.reedy

versions: - Python 3.1
2012-11-18 19:06:11roger.serwysetfiles: + editor_token_error_rev1.patch

messages: + msg175904
2012-11-18 18:38:22serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg175901
2012-11-18 17:39:54roger.serwysetfiles: + sample_token_error.py

messages: + msg175896
2012-11-18 17:37:09roger.serwycreate