classification
Title: ttk.Treeview "unmatched open brace in list"
Type: behavior Stage: resolved
Components: Tkinter Versions: Python 3.4, Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Bryan.Oakley, Stefan.Stuhr, asvetlov, eric.smith, ezio.melotti, gpolo, ned.deily, python-dev, serhiy.storchaka, zach.ware
Priority: normal Keywords: patch

Created on 2012-09-04 21:41 by Bryan.Oakley, last changed 2013-01-15 16:07 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
tkinter_quoting.patch serhiy.storchaka, 2012-11-10 15:36 review
tkinter_quoting_3.patch serhiy.storchaka, 2012-12-12 09:42 Patch for 3.x review
tkinter_quoting_3-2.7.patch serhiy.storchaka, 2012-12-12 09:44 Patch for 2.7 review
Messages (16)
msg169839 - (view) Author: Bryan Oakley (Bryan.Oakley) Date: 2012-09-04 21:41
If you try to insert an item into the treeview, give it a tuple of values for the "values" attribute, and one of those values has unbalanced braces, you'll get an error "unmatched open brace in list"

To reproduce:

import Tkinter as tk
import ttk

root = tk.Tk()
tree = ttk.Treeview(root)
tree.insert("","end",values=("one","two","bam! {"))

root.mainloop()
msg169862 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-09-05 07:15
What behavior do you expect?  By accident, I found your reply to this StackOverflow question.  I take it from that that you think tkinter should be quoting curly braces found in strings.  If that is what you mean, can you specify the set of characters that need to be escaped and in what contexts?

http://stackoverflow.com/questions/11566299/unmatched-open-brace-error-when-adding-item-to-ttk-treeview
msg169866 - (view) Author: Bryan Oakley (Bryan.Oakley) Date: 2012-09-05 11:08
What behavior do I expect? I expect it to not throw an error. I expect whatever string I give to be inserted into the widget unadulterated (ie: if I give the string "foo {" I expect to see "foo {" in the widget). 

Tkinter is effectively telling me "you have a Tcl syntax error". Since I'm programming in python I should be insulated from that, particularly since the error comes internally after Tkinter transforms my data. 

How Tkinter does it under the hood, I don't care. Tkinter should make sure that the data it passes to the Tcl interpreter is well-formed.
msg169870 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2012-09-05 13:06
I agree with Bryan. Further, if the string is being interpreted as Tcl, then this strikes me as a possible injection attack vector (although I'll admit to not having looked at the code to see how the Tcl code is being used and/or interpreted).
msg169877 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-09-05 15:35
I agree as well.  The bigger question is, as Eric suggests, is this the only such case when mapping between Python and Tcl elements?  Since Bryan is a well-known Tcl and Tkinter expert, his insights on that would be very useful.
msg169938 - (view) Author: Bryan Oakley (Bryan.Oakley) Date: 2012-09-06 20:31
I gave myself an hour or so to play around with this, and the crux of the matter seems to be in the function `_format_optdict()` which converts a dictionary of options and values into something suitable to pass to `tk.call()`. However, I think the same bug is in other `_format*` functions as well, it's just that their nature is such that they have much less of a chance to be passed weird data.

`_format_optdict` has some code that does a half-hearted attempt at handling values that are tuples, such as the case with the "values" attribute of the ttk.Treeview widget. However, all it does is protect values that have a space, by surrounding the value with curly braces. Hence, when the value itself has a curly brace, tcl throws the "unmatched open brace" error. 

What is needed is to create a bona fide tcl list element according to the rules of Tcl. I tried a hack where I simply escaped all problem characters, so instead of returning `{foo bar}` the function returns `foo\ bar`. This seemed to work, at least for the tiny bit of testing that I did. Another solution might be to do something like tk.call("list",*the_tuple), though sadly, `_format_optdict` is a function rather than a method so it doesn't have access to the tcl interpreter. 

What I think ttk needs (and may already exist somewhere in the Tkinter world; I haven't looked...) is a function that takes a tuple and converts it to a canonical list. Then, the places that do something ad hoc can all call this one function. 

For more information on the gory details of the string representation of a list see http://www.tcl.tk/cgi-bin/tct/tip/407.html
msg175194 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2012-11-08 22:04
Just tested on 3.3 and this still happens with that version of Tkinter, which should mean the same happens in 3.2 and 3.4.
msg175207 - (view) Author: Stefan Stuhr (Stefan.Stuhr) Date: 2012-11-08 23:11
I think this is a case of faulty over-engineering in the ttk module. The following works just fine (imports converted to Python3):

import tkinter as tk
import tkinter.ttk as ttk

root = tk.Tk()
tree = ttk.Treeview(root, columns="1 2 3")
tree.tk.call(tree, "insert", "", "end" , "-values", ("one","two","bam! {"))
tree.grid()

root.mainloop()
msg175283 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-10 15:36
Here is a patch for right quoting of arguments before concatenation.  Thanks Bryan Oakley for the analysis.
msg175795 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-17 18:07
Patch updated.
msg177367 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-12 09:42
Patch updated. Added additional tests for ttk.Combobox (see issue11290).
msg177368 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-12 09:44
Patch for 2.7 added.
msg177963 - (view) Author: Guilherme Polo (gpolo) * (Python committer) Date: 2012-12-23 00:41
I consider myself the main responsible for this bug, so if the current patch (which I didn't look at) passes all tests, doesn't slow things up (unlikely, I would think) and also works with some more involved styling code then I'm fine with it.

These formatting functions were added to the ttk module to make it easier to create and adjust styles using Python. It wouldn't be possible without them unless the person basically coded in Tcl while in Python. These functions ended up being used in other places (as was noticed here) for convenience. All in all, I'm glad someone else is using the module, reporting, and improving it.
msg178600 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-30 18:46
Have you reviewed the patch? Is it good?
msg180027 - (view) Author: Roundup Robot (python-dev) Date: 2013-01-15 16:04
New changeset 411bb75be5d1 by Serhiy Storchaka in branch '3.2':
Issue #15861: tkinter now correctly works with lists and tuples containing
http://hg.python.org/cpython/rev/411bb75be5d1

New changeset 927352d7e994 by Serhiy Storchaka in branch '3.3':
Issue #15861: tkinter now correctly works with lists and tuples containing
http://hg.python.org/cpython/rev/927352d7e994

New changeset 340e97ebe911 by Serhiy Storchaka in branch 'default':
Issue #15861: tkinter now correctly works with lists and tuples containing
http://hg.python.org/cpython/rev/340e97ebe911

New changeset 917ae14831ec by Serhiy Storchaka in branch '2.7':
Issue #15861: tkinter now correctly works with lists and tuples containing
http://hg.python.org/cpython/rev/917ae14831ec
msg180028 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-15 16:06
Fixed. Thank you for report, Bryan Oakley.
History
Date User Action Args
2013-01-15 16:07:08serhiy.storchakasetstatus: open -> closed
2013-01-15 16:06:40serhiy.storchakasetresolution: fixed
messages: + msg180028
stage: patch review -> resolved
2013-01-15 16:04:35python-devsetnosy: + python-dev
messages: + msg180027
2012-12-30 18:46:10serhiy.storchakasetmessages: + msg178600
2012-12-29 21:53:16serhiy.storchakasetassignee: serhiy.storchaka
2012-12-23 00:41:28gpolosetnosy: + gpolo
messages: + msg177963
2012-12-12 09:46:01serhiy.storchakalinkissue11290 superseder
2012-12-12 09:44:16serhiy.storchakasetfiles: + tkinter_quoting_3-2.7.patch

messages: + msg177368
2012-12-12 09:42:38serhiy.storchakasetfiles: + tkinter_quoting_3.patch

messages: + msg177367
2012-12-12 09:23:17serhiy.storchakasetfiles: - tkinter_quoting_2.patch
2012-11-17 18:07:04serhiy.storchakasetfiles: + tkinter_quoting_2.patch

messages: + msg175795
2012-11-11 16:34:08serhiy.storchakasetstage: needs patch -> patch review
2012-11-10 15:36:47serhiy.storchakasetfiles: + tkinter_quoting.patch
keywords: + patch
messages: + msg175283
2012-11-08 23:11:12Stefan.Stuhrsetnosy: + Stefan.Stuhr
messages: + msg175207
2012-11-08 22:10:02serhiy.storchakasetnosy: + serhiy.storchaka

stage: needs patch
2012-11-08 22:04:23zach.waresetnosy: + zach.ware
messages: + msg175194
2012-11-08 22:02:36zach.waresetversions: + Python 3.2, Python 3.3, Python 3.4
2012-09-06 20:31:20Bryan.Oakleysetmessages: + msg169938
2012-09-05 15:35:03ned.deilysetmessages: + msg169877
2012-09-05 13:06:16eric.smithsetnosy: + eric.smith
messages: + msg169870
2012-09-05 11:37:17ezio.melottisetnosy: + ezio.melotti
2012-09-05 11:08:17Bryan.Oakleysetmessages: + msg169866
2012-09-05 07:15:53ned.deilysetnosy: + asvetlov, ned.deily
messages: + msg169862
2012-09-04 21:41:42Bryan.Oakleycreate