classification
Title: IDLE "AttributeError: 'NoneType' object has no attribute 'index'" from Delegator.py
Type: behavior Stage: patch review
Components: IDLE Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: roger.serwy Nosy List: Todd.Rovito, benjamin.peterson, ezio.melotti, georg.brandl, larry, ned.deily, orsenthil, python-dev, rhettinger, roger.serwy, terry.reedy
Priority: normal Keywords: patch

Created on 2013-04-02 00:55 by rhettinger, last changed 2013-04-07 17:43 by roger.serwy. This issue is now closed.

Files
File name Uploaded Description Edit
stop_colorizer.patch roger.serwy, 2013-04-02 03:31 review
Messages (18)
msg185784 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-04-02 00:55
Running ILDE with 2.7.4 RC 1 on Mac OS X 10.8 and with ActiveTCL 8.5.13 using "python -m idlelib.idle", I object the following traceback messages.  I'm unsure what events are triggering it:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1470, in __call__
    return self.func(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 531, in callit
    func(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/idlelib/ColorDelegator.py", line 153, in recolorize
    self.recolorize_main()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/idlelib/ColorDelegator.py", line 186, in recolorize_main
    next = self.index(mark + "+%d lines linestart" %
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/idlelib/Delegator.py", line 10, in __getattr__
    attr = getattr(self.delegate, name) # May raise AttributeError
AttributeError: 'NoneType' object has no attribute 'index'
msg185787 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-04-02 01:40
Raymond,
   Is then when you simply startup IDLE or are you opening a Python file to edit? The only reason why I ask about editing a file is because ColorDelegator is used to by the EditorWindow to perform the syntax hi-lighting (I think).  I have a few Macs so I can try and duplicate.  In addition I added a few people to the nosy list that might know better than I do what the problem is.  Thanks for the report.
msg185793 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-04-02 02:16
Setting as 2.7.4 release blocker pending further analysis. Roger, any ideas?  Would having the contents of Raymond's .idlerc files be of help?  Raymond, does IDLE terminate when the exception occurs or does it keep running?
msg185796 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-04-02 03:02
Do you get the same traceback with "python -m idlelib"?
msg185797 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-04-02 03:06
"python -m idlelib" isn't implemented for 2.7.x, is it?
msg185800 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-04-02 03:31
It's a "race condition" due to ColorDelegator calling .update() in its recolorize method. The call to .update() flushes pending events, including callbacks into Python code that can change out the ColorDelegator in the Percolator chain. 

Here's a way to trigger this bug reliably. First we need an unreasonably large file that will make the syntax highlighting routine take a while. This code will generate such a file:


fid = open('/tmp/bigfile.py', 'w')
L = '\n'.join(map(lambda x: "#%i\n'%i'\n" % (x,x), range(10000)))
fid.writelines(L)
fid.close()


These next two steps need to be performed quickly to trigger the "race condition."

$ python -m idlelib.idle -e /tmp/bigfile.py
As soon as it loads, press Ctrl+Shift+S to bring up the Save-As dialog. 

You can take your time now. Overwrite the file. This triggers the filename_change_hook to call ResetColorizer, which removes the current ColorDelegator from the Percolator chain and inserts a new one. Wait a moment for the traceback to arrive.

The attached patch fixes the issue. When the ColorDelegator is removed by Percolator's removefilter, the delegate is set to None. The patch detects that condition and then sets flags to stop the colorizer from continuing.


In testing for this bug, I encountered another one. Pressing Alt-F4 quickly after the window loading bigfile appears triggers another traceback. I'll open a separate issue for that.
msg185801 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-04-02 03:37
Yes, idlelib.__main__.py was only added to 3.3.
msg185808 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-04-02 04:37
Removing Geoge and Larry from nosy. I'm not sure how they got added. I didn't click on anything in the Nost List box.

Ned, did you mean to add Benjamin to the nosy list?
msg185809 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-04-02 04:42
Is there an automatic nosy-list adder routine in the bug-tracker that I should know about? Georg and Larry were re-added to nosy even after I removed them. (I'm sorry for the spam.)
msg185810 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-04-02 04:43
> Raymond, does IDLE terminate when the exception occurs 
> or does it keep running?

It keeps running.
msg185812 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-04-02 04:54
When an issue is marked as "release blocker", the release managers of all affected versions are automatically added to the issue.  I'm the release manager of 3.4, Georg is for 3.3.
msg185814 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-04-02 04:58
> Is there an automatic nosy-list adder routine in the bug-tracker that
> I should know about? 

Release managers are added automatically when an issue is set as release blocker and one of the releases they manage is selected in the "Versions" field.
msg185816 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2013-04-02 05:12
I dont have ActiveState TCL, but I could use IDLE with the default libs with my Macbook Pro (OS X10.7), followed the steps given to reproduce the error and unfortunately could not.
msg185818 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-04-02 05:57
I'm able to reliably reproduce the exception using your test procedure, Roger, using either current head of 3.3 or 2.7, and am not able to reproduce it with stop_colorizer.patch applied.  LGTM.  So it seems to me the next questions are:

1. Raymond, does this seem like a plausible scenario for the failures you've seen?  And if you apply the patch, do you see any further failures?

2. Roger, is this issue new to 2.7.4, 3.2.4, or 3.3.1?  How likely do you think it will be seen by users?  And what is your recommendation to the release managers about including the patch in any or all of the pending maintenance releases, understanding that the options are to cherry-pick into the final and hope for the best, cherry-pick it and do an RC2, or defer the fix to the next maintenance releases?
msg185819 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-04-02 06:43
On 04/02/2013 12:57 AM, Ned Deily wrote:
> 2. Roger, is this issue new to 2.7.4, 3.2.4, or 3.3.1?
No, this bug has been baked into IDLE since the beginning. See line 215 
http://hg.python.org/cpython/file/30713d5288df/Lib/idlelib/ColorDelegator.py
>   How likely do you think it will be seen by users?
For Windows users using pythonw.exe, it would cause IDLE to immediately 
terminate due to issue13582. On Linux and Mac, the errors would write to 
stderr, though not likely be visible to the end user. On Linux and Mac 
the error is not "harmful" since IDLE still remains perfectly usable. 
The error does not leave IDLE in an inconsistent state.
>   And what is your recommendation to the release managers about including the patch in any or all of the pending maintenance releases, understanding that the options are to cherry-pick into the final and hope for the best, cherry-pick it and do an RC2, or defer the fix to the next maintenance releases?
I'm ok with cherry-picking it into the final, but in the interest of 
simplicity, I'd recommend deferring it to the next maintenance release. 
There are other bugs, like #8900 that are easier to trigger than this one.
msg185829 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-04-02 12:17
Thank you for the analysis.
msg186232 - (view) Author: Roundup Robot (python-dev) Date: 2013-04-07 17:42
New changeset ae05d841bea1 by Roger Serwy in branch '2.7':
#17613: Prevent traceback when removing syntax colorizer in IDLE.
http://hg.python.org/cpython/rev/ae05d841bea1

New changeset 8b793a946acb by Roger Serwy in branch '3.3':
#17613: Prevent traceback when removing syntax colorizer in IDLE.
http://hg.python.org/cpython/rev/8b793a946acb

New changeset b45675223885 by Roger Serwy in branch 'default':
#17613: merge with 3.3.
http://hg.python.org/cpython/rev/b45675223885
msg186233 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-04-07 17:43
Thank you everyone for your patience. I am closing this issue as fixed.
History
Date User Action Args
2013-04-07 17:43:34roger.serwysetstatus: open -> closed
assignee: roger.serwy
resolution: fixed
messages: + msg186233
2013-04-07 17:42:03python-devsetnosy: + python-dev
messages: + msg186232
2013-04-02 12:17:45benjamin.petersonsetpriority: release blocker -> normal

messages: + msg185829
2013-04-02 09:45:48ezio.melottisetversions: - Python 3.2
2013-04-02 06:43:41roger.serwysetmessages: + msg185819
2013-04-02 05:57:35ned.deilysetmessages: + msg185818
2013-04-02 05:12:48orsenthilsetnosy: + orsenthil
messages: + msg185816
2013-04-02 04:58:56ezio.melottisetnosy: + ezio.melotti
messages: + msg185814
2013-04-02 04:54:30larrysetmessages: + msg185812
2013-04-02 04:43:18rhettingersetmessages: + msg185810
2013-04-02 04:42:04roger.serwysetmessages: + msg185809
2013-04-02 04:37:02roger.serwysetnosy: georg.brandl, rhettinger, terry.reedy, larry, benjamin.peterson, ned.deily, roger.serwy, Todd.Rovito
messages: + msg185808
2013-04-02 03:37:56terry.reedysetmessages: + msg185801
2013-04-02 03:31:44roger.serwysetfiles: + stop_colorizer.patch

versions: + Python 3.2, Python 3.3, Python 3.4
keywords: + patch
nosy: + georg.brandl, larry

messages: + msg185800
stage: patch review
2013-04-02 03:06:59ned.deilysetmessages: + msg185797
2013-04-02 03:02:36terry.reedysetmessages: + msg185796
2013-04-02 02:16:34ned.deilysetpriority: normal -> release blocker
title: IDLE error -> IDLE "AttributeError: 'NoneType' object has no attribute 'index'" from Delegator.py
nosy: + benjamin.peterson

messages: + msg185793
2013-04-02 01:40:20Todd.Rovitosetnosy: + terry.reedy, ned.deily, roger.serwy, Todd.Rovito
messages: + msg185787
2013-04-02 00:55:15rhettingercreate