Title: IDLE 3 crashes processing byte strings with invalid hex escape sequences
Type: Stage: resolved
Components: IDLE Versions: Python 3.2, Python 3.3
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: Claudiu.Popa, ned.deily, python-dev, terry.reedy
Priority: normal Keywords: patch

Created on 2010-09-16 08:55 by Claudiu.Popa, last changed 2011-09-14 22:03 by ned.deily. This issue is now closed.

File name Uploaded Description Edit
idle.patch Claudiu.Popa, 2011-09-14 11:08
Messages (8)
msg116516 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2010-09-16 08:55

While trying to find a way for extracting strange characters in an ascii file, I stumbled upon some strange behaviour of IDLE, which exits without warning after running the following regex:

re.findall(b"\x.{2}", b"sdds\xd8")

In Python 2.6, this won't happen. This is my version of Python 3.1:
Python 3.1 (r31:73574, Jun 26 2009, 20:21:35).
msg116753 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-09-18 03:40
Verified as IDLE specific on WinXP, 3.1.2

With interpreter, I get
>>> import re; re.findall(b"\x.{2}", b"sdds\xd8")
ValueError: invalid \x escape

In an IDLE Shell window, the window immediately disappears when I hit return and the taskbar button disappears about a third of a second later.
msg144016 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2011-09-14 08:58
Hello. This happens with this version of IDLE too: 

Python 3.2.1 (default, Jul 10 2011, 21:51:15) [MSC v.1500 32 bit (Intel)] on win32. 
I could write a patch if I knew where to start.
msg144019 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2011-09-14 10:33
I found the culprit. I wrote b"\x" in IDLE and the window immediatly disappeared. Then, to traceback the problem, I started a python shell and typed the following lines. It seems that in showsyntaxerror, value is different that what was expected. 

Python 3.2.1 (default, Jul 10 2011, 21:51:15) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import idlelib.PyShell
>>> idlelib.PyShell.main()
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python32\lib\", line 63, in runsource
    code = self.compile(source, filename, symbol)
  File "C:\Python32\lib\", line 168, in __call__
    return _maybe_compile(self.compiler, source, filename, symbol)
  File "C:\Python32\lib\", line 82, in _maybe_compile
    code = compiler(source, filename, symbol)
  File "C:\Python32\lib\", line 133, in __call__
    codeob = compile(source, filename, symbol, self.flags, 1)
ValueError: invalid \x escape

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python32\lib\tkinter\", line 1399, in __call__
    return self.func(*args)
  File "C:\Python32\lib\idlelib\", line 166, in handler
    r = l[i](event)
  File "C:\Python32\lib\idlelib\", line 1126, in enter_callback
  File "C:\Python32\lib\idlelib\", line 1167, in runit
    more = self.interp.runsource(line)
  File "C:\Python32\lib\idlelib\", line 617, in runsource
    return InteractiveInterpreter.runsource(self, source, filename)
  File "C:\Python32\lib\", line 66, in runsource
  File "C:\Python32\lib\idlelib\", line 654, in showsyntaxerror
    msg = value.msg or "<no detail available>"
AttributeError: 'ValueError' object has no attribute 'msg'
msg144020 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2011-09-14 11:06
I've attached a patch, hope it is ok.
msg144021 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2011-09-14 11:07
Wrong patch.
msg144055 - (view) Author: Roundup Robot (python-dev) Date: 2011-09-14 21:57
New changeset e74860883a9c by Ned Deily in branch '3.2':
Issue #9871: Prevent IDLE 3 crash when given byte stings

New changeset fe6c7771c25c by Ned Deily in branch 'default':
Issue #9871: Prevent IDLE 3 crash when given byte stings
msg144056 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-09-14 22:03
Thanks for your suggested patch!  A similar change was needed in idle lib/ to handle the same case for script files in edit windows.  Applied in 3.2 (for 3.2.3) and default (for 3.3).
Date User Action Args
2011-09-14 22:03:26ned.deilysetstatus: open -> closed

assignee: ned.deily

title: IDLE dies when using some regex -> IDLE 3 crashes processing byte strings with invalid hex escape sequences
nosy: + ned.deily
versions: + Python 3.3, - Python 3.1
messages: + msg144056
resolution: fixed
stage: test needed -> resolved
2011-09-14 21:57:30python-devsetnosy: + python-dev
messages: + msg144055
2011-09-14 11:08:17Claudiu.Popasetfiles: + idle.patch
2011-09-14 11:07:56Claudiu.Popasetmessages: + msg144021
2011-09-14 11:07:39Claudiu.Popasetfiles: - idle.patch
2011-09-14 11:06:31Claudiu.Popasetfiles: + idle.patch
keywords: + patch
messages: + msg144020
2011-09-14 10:33:46Claudiu.Popasetmessages: + msg144019
2011-09-14 08:58:17Claudiu.Popasetmessages: + msg144016
2010-09-18 03:40:56terry.reedysetversions: + Python 3.2
nosy: + terry.reedy

messages: + msg116753

stage: test needed
2010-09-16 08:55:13Claudiu.Popacreate