classification
Title: IDLE: source line in editor doesn't highlight when debugging
Type: behavior Stage: resolved
Components: IDLE Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: roger.serwy Nosy List: Rich.Rauenzahn, Todd.Rovito, dzabel, loewis, python-dev, roger.serwy, terry.reedy
Priority: normal Keywords: patch

Created on 2012-02-28 04:01 by Rich.Rauenzahn, last changed 2014-08-16 01:10 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
win_highlighting.patch roger.serwy, 2013-04-12 05:30 review
Messages (31)
msg154533 - (view) Author: Rich Rauenzahn (Rich.Rauenzahn) Date: 2012-02-28 04:01
Using 64bit python for windows downloaded from python.org on 64bit windows 7.

Python Version 3.2.2
Tk version 8.5
IDLE version 3.2.2

When stepping through code the corresponding line in the editor does not highlight with the code steps.  The windows does update the contents, so it appears to be tracking, but just fails to highlight the line.

Double clicking on the line in the debugger will go ahead and highlight it.  My settings are all default, and I've double checked the color schemes in the "highlighting" dialog.
msg154566 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-02-28 19:28
Is the "Source" check box in the Debug Control window checked?
msg154571 - (view) Author: Rich Rauenzahn (Rich.Rauenzahn) Date: 2012-02-28 20:31
Yes, the source box was checkmarked.

Not the first one to encounter this as well:

http://python.6.n6.nabble.com/IDLE-not-highlighting-current-line-while-debugging-on-Win-td2113345.html
msg154591 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-02-29 02:10
I am not seeing this problem under Ubuntu, but I do see this problem on Vista. It looks like the "sel" tags get hidden when a window loses focus under Windows.
msg183793 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-09 04:14
I closed #17382 as a duplicate of this. The OP, Dirk, only had problem on Windows xp, not on Ubuntu 3.2 and 3.3. I see the same problem on 3.3 win 7.

Roger, do you think this is a windows, tkinter, or idle problem?
msg183833 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-03-09 16:54
I think it's a problem on Tk on Windows. The painting of the selection 
highlight goes away when a window loses focus.
msg183851 - (view) Author: Dirk Zabel (dzabel) Date: 2013-03-09 22:13
If this is a Tk problem on windows, are there any chances to get it fixed? The Tk version installed with python 3.3 seems to be 8.5.11, while there are newer Tk versions available on www.activestate.com (8.5.13 and 8.6.0). But I don't know how to connect a stand-alone version of Tcl/Tk with Python, and I don't want to mess up my Python installation.
msg183853 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-09 22:45
Martin, is there any way to test if tcl/tk 8.5.13 and/or 8.6.0 fix this windows-only tk issue?
msg185516 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-29 20:55
I was going to try Python 3.4 and TK 8.6 on Windows and see what happens.
msg185535 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-30 01:32
I tried both TCL/TK 8.5.13 and TCL/TK 8.6 with the latest Python 3.4 on Windows 7 the editor window never showed a line as I stepped through the debugger.  I am going to try in Mac/Linux to make sure I am not crazy that a line in the editor window does indicate where the debugger is.  Then I intend to look in the bug tracker on TCL/TK and possibly post a question for help.  On Windows the initial line does look funny on a line with a quoted string such as print("hello world") the yellow does not cover the quoted string.
msg185536 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-30 03:15
Before I forget here are the general steps I followed to get TCL/TK 8.5.13 and 8.6 to work.  For TCL/TK 8.6 I had to change the actual Visual Studio 2010 project. 

Generally you have to follow the steps in readme.txt located in PCBuild of the source tree.  readme.txt references a file Tools\buildbot\external.bat which has the step by step instructions on how to build TCL/TK.  Skip the part about going to svn.python.org/external because neither 8.6 or 8.5.13 have been added yet.  So I downloaded the code from the sourceforge.net TCL/TK site.  This post explains a step that must be performed which is not documented in any of the references above:
http://mail.python.org/mailman/private/core-mentorship/2013-March/001593.html

Finally for TCL/TK 8.6 I had to change the visual studio project to use the right .lib file for TCL/TK.  Start Visual Studio 2010 right click _tkinter project then select properties.  Under the Linker/Input option set Additional Dependencies to c:\prog\cpython\tcltk\lib\tcl86tg.lib and c:\prog\cpython\tcltk\lib\tk86tg.lib

erase the old values.  Now rebuild and everything should just work (c) don't forget to manually copy the DLL files to the PCBuild directory.
msg185539 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-30 03:35
I have confirmed that Linux and Mac work great but Windows fails to highlight the current line in the editor window.  Next I will try and find/file a bug with the TCL/TK folks.....
msg185578 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-30 17:31
Here is a backtrace from PDB:

-> self.sync_source_line()
  /Volumes/SecurePython3/cpython/py34/Lib/idlelib/Debugger.py(211)sync_source_line()
-> self.flist.gotofileline(filename, lineno)
  /Volumes/SecurePython3/cpython/py34/Lib/idlelib/FileList.py(46)gotofileline()
-> edit.gotoline(lineno)
> /Volumes/SecurePython3/cpython/py34/Lib/idlelib/EditorWindow.py(694)gotoline()
-> self.center()

The offending code seems to be in EditorWindow.py:gotoline()

    def gotoline(self, lineno):
        if lineno is not None and lineno > 0:
            self.text.mark_set("insert", "%d.0" % lineno)
            self.text.tag_remove("sel", "1.0", "end")
            self.text.tag_add("sel", "insert", "insert +1l")
            self.center()

Next I am going to write a small program to try and reproduce the bug so I can file with the TCL/TK folks.
msg185582 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-30 18:21
I created a small test program trying to reproduce the problem on Windows 7, Python 3.4, and TK 8.6.  Unfortunately it works fine and each line is highlighted as a user presses ctrl-B.  I got the select code straight from IDLE.  Any other ideas Roger?  This might be an IDLE bug after all.

from tkinter import *

class SelectTest:
    def __init__(self):
        self.mainwin = Tk()
        # Create a text widget
        # idle creates the text widget like this
        #self.text = text = MultiCallCreator(Text)(text_frame, **text_options)
        self.textbox = Text(self.mainwin, width=80, height=10)
        self.textbox.pack()

        # Add some text
        self.textbox.insert(INSERT, "line 1: Select some text\n")
        self.textbox.insert(INSERT, "line 2: Select some text\n")
        self.textbox.insert(INSERT, "line 3: Select some text\n")
        self.textbox.insert(INSERT, "line 4: Select some text\n")

        # Add the binding
        self.textbox.bind("<Control-Key-b>", self.select_just_like_idle)
        # just in case caps lock is on
        self.textbox.bind("<Control-Key-B>", self.select_just_like_idle)
        self.lineno = 1
	
    def select_just_like_idle(self, event):
        print("Select just like idle was called")
        if self.lineno is not None and self.lineno > 0:
            self.textbox.mark_set("insert", "%d.0" % self.lineno)
            self.textbox.tag_remove("sel", "1.0", "end")
            self.textbox.tag_add("sel", "insert", "insert +1l")
            self.lineno = self.lineno + 1


if __name__ == "__main__":
    # Start the program
    selectIDLETest = SelectTest()
    selectIDLETest.mainwin.mainloop()
msg185608 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-03-31 03:44
It's definitely a "bug" with Tk. Whenever the Text widget loses focus, the selection highlighting goes away. The following example code brings up a focused text box with selected text. Clicking the button switches focus to the button itself and then back to the text widget. During that focus change, the selection highlight goes away.

from tkinter import *
main = Tk()
text = Text(main, width=40, height=10, wrap="char")
text.pack()
text.insert(INSERT, "".join(map(str, range(100))))
text.tag_add(SEL, "1.0", "end")
text.focus_set()
def jump():
    text.after(500, btn.focus_set)
    text.after(1000, text.focus_set)
btn = Button(main, text="Click me", command=jump)
btn.pack()
main.mainloop()


One possible solution would be to use the "<FocusOut>" and "<FocusIn>" events to substitute the "sel" tag with a custom tag so that the highlight remains when losing focus, and then replace it with the "sel" tag when gaining focus.
msg185631 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-31 17:09
Roger,
  You are a genius!!!!!  The example program duplicates the bug exactly.  It works on Mac (I assume Linux but I will test on Linux) and it does not work on correctly on Windows.  On Windows as soon as the text widget looses focus then the hi-light disappears.  I will use this code and file a bug report for TCL/TK and post on the tkinter mailing list.  Thank you I was stuck.
msg185632 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-03-31 17:27
Bug report has been filed with Tk here:
https://sourceforge.net/tracker/?func=detail&aid=3609608&group_id=12997&atid=112997
msg186061 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-04-05 01:57
I posted this message on tinter discuss email list:
http://mail.python.org/pipermail/tkinter-discuss/2013-March/003415.html
msg186608 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-04-12 05:30
Todd, thank you for being proactive with the Tcl/Tk community. Hopefully they will offer a fix in their next version.

In the meanwhile, here's a patch that works around the problem on Windows. The purpose of getting the highlight configuration at each FocusOut event is in case the current theme changes.
msg186610 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-04-12 05:51
I forgot to mention that the idea for replacing the "sel" tags is based on an idea from Sarah's patch from #17511.
msg188655 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-05-07 13:56
I'm pinging this issue to see if anyone has had any problems with the Windows-specific workaround for highlighting the selection tags. Issue17511 depends on this fix.
msg188751 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-05-09 05:09
64 bit Win 7 with 32 bit debug build.
Patch imported cleanly with all 3 branches. I cannot currently test 2.7. On 3.3 and 3.4, debugger worked fine relative to this issue: editor window highlight tracked line displayed in debugger as far as I checked.
Normal selection and Shell Go to file/line highlight seem to work normally. Normal color marking seems not affected. I cannot think of any more manual tests. So if someone else checks 2.7 and it is also ok, I think you should apply this, so if there is any problem, it can show up in other usage. We can deal with a patched tk if and when it happens.
msg189092 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-05-13 04:02
I'm waiting until after the next wave of maitenance releases before I apply this patch. If anyone feels strongly that it should be applied now, let me know.

I applied the patch to the latest 2.7.4 64-bit release version of Python and it worked.
msg189096 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-05-13 04:32
Roger,
    If you and Terry tested I would apply now so it makes it into 2.7.5.  Why not?  Right now the debugger in Windows doesn't highlight and I am sure that has to drive people crazy.  But if you feel it needs more testing maybe you should let it bake some more?
msg189192 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-05-14 02:30
It won't make it in 2.7.5. Benjamin tagged the 2.7.5 release a couple of days ago.

I'll apply this later tonight.
msg189194 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-05-14 03:14
On second thought, I'll wait until after the releases so that Misc/NEWS gets populated properly.
msg189718 - (view) Author: Roundup Robot (python-dev) Date: 2013-05-21 03:18
New changeset 5ae830ff6d64 by Roger Serwy in branch '2.7':
#14146: Highlight source line while debugging on Windows.
http://hg.python.org/cpython/rev/5ae830ff6d64

New changeset 3735b4e0fc7c by Roger Serwy in branch '3.3':
#14146: Highlight source line while debugging on Windows.
http://hg.python.org/cpython/rev/3735b4e0fc7c

New changeset b56ae3f878cb by Roger Serwy in branch 'default':
#14146: merge with 3.3.
http://hg.python.org/cpython/rev/b56ae3f878cb
msg189719 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-05-21 03:21
I committed the Tk workaround for the Windows platform. I'm leaving this issue as pending with a resolution of later in case Tk developers address the bug report mentioned in msg185632.

If anyone else wishes to close it, feel free.
msg190271 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-05-29 04:24
This appears fixed for not, even if not the way we would like. Thanks.
If a future tk changes, a new patch will be version-dependent and that will be a new issue.
msg225376 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-08-16 00:08
Regardless of what I said in the previous message, highlighting of found text is NOT working in any current release -- 2.7.8, 3.3.5 (final release) and 3.4.1.  Mark Lawrence opened #22179. I propose there to use the 'found' highlight, as used in the replace dialog. for the find dialog also.
msg225380 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-08-16 01:10
Sorry, wrong issue (should have been #17511). Debugger source line highlighting works fine.
History
Date User Action Args
2014-08-16 01:10:20terry.reedysetmessages: + msg225380
2014-08-16 00:08:17terry.reedysetmessages: + msg225376
2013-05-29 04:24:23terry.reedysetstatus: pending -> closed
resolution: later -> fixed
messages: + msg190271
2013-05-21 03:21:03roger.serwysetstatus: open -> pending
resolution: later
messages: + msg189719

stage: commit review -> resolved
2013-05-21 03:18:02python-devsetnosy: + python-dev
messages: + msg189718
2013-05-14 03:14:34roger.serwysetmessages: + msg189194
2013-05-14 02:30:29roger.serwysetmessages: + msg189192
stage: patch review -> commit review
2013-05-13 04:32:51Todd.Rovitosetmessages: + msg189096
2013-05-13 04:02:39roger.serwysetmessages: + msg189092
2013-05-09 05:09:33terry.reedysetmessages: + msg188751
2013-05-07 13:56:25roger.serwysetassignee: roger.serwy
messages: + msg188655
2013-04-12 05:51:25roger.serwysetmessages: + msg186610
2013-04-12 05:50:19roger.serwylinkissue17511 dependencies
2013-04-12 05:30:42roger.serwysetfiles: + win_highlighting.patch
versions: + Python 2.7, - Python 3.2
messages: + msg186608

keywords: + patch
stage: patch review
2013-04-05 01:57:49Todd.Rovitosetmessages: + msg186061
2013-03-31 17:27:10Todd.Rovitosetmessages: + msg185632
2013-03-31 17:09:04Todd.Rovitosetmessages: + msg185631
2013-03-31 03:44:56roger.serwysetmessages: + msg185608
2013-03-30 18:21:16Todd.Rovitosetmessages: + msg185582
2013-03-30 17:31:27Todd.Rovitosetmessages: + msg185578
2013-03-30 03:35:13Todd.Rovitosetmessages: + msg185539
2013-03-30 03:15:16Todd.Rovitosetmessages: + msg185536
2013-03-30 01:32:46Todd.Rovitosetmessages: + msg185535
2013-03-29 20:55:57Todd.Rovitosetmessages: + msg185516
2013-03-29 20:55:04Todd.Rovitosetnosy: + Todd.Rovito
2013-03-09 22:45:05terry.reedysetnosy: + loewis
messages: + msg183853
2013-03-09 22:13:54dzabelsetnosy: + dzabel
messages: + msg183851
2013-03-09 16:54:56roger.serwysetmessages: + msg183833
2013-03-09 04:14:17terry.reedysetmessages: + msg183793
versions: + Python 3.3, Python 3.4
2013-03-09 04:09:44terry.reedylinkissue17382 superseder
2012-03-02 21:34:52terry.reedysetnosy: + terry.reedy
2012-02-29 02:10:38roger.serwysetmessages: + msg154591
2012-02-28 20:31:07Rich.Rauenzahnsetmessages: + msg154571
2012-02-28 19:28:21roger.serwysetnosy: + roger.serwy
messages: + msg154566
2012-02-28 04:01:48Rich.Rauenzahncreate