classification
Title: PhotoImage(data=...) apparently has to be UTF-8 or Base-64 encoded
Type: behavior Stage: resolved
Components: Documentation, Tkinter Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: 21605 Superseder:
Assigned To: serhiy.storchaka Nosy List: docs@python, gpolo, loewis, martin.panter, python-dev, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2014-05-26 05:52 by martin.panter, last changed 2014-07-31 07:33 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
tkinter_bytes.patch serhiy.storchaka, 2014-06-01 13:05 review
tkinter_bytes-2.7.patch serhiy.storchaka, 2014-06-01 13:11 Patch for 2.7. review
tkinter_bytes-2.7_2.patch serhiy.storchaka, 2014-07-30 15:48 review
Messages (11)
msg219133 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-05-26 05:52
At the bottom of the “tkinter” doc page it mentions the “data” option is available. However I was unable to get it to work well when passing a plain bytes() string in. It seems the bytes() string gets interpreted as UTF-8 somewhere along the line, although the underlying TCL library apparenly handles byte strings fine itself. Passing binary GIF and PNG data in Python would usually produce strange exceptions, crashes, and blank images for me.

I found this message with a simple patch which might be useful, though I’m not familiar with the code involved to understand the implications:
https://mail.python.org/pipermail/tkinter-discuss/2012-April/003108.html

Even if that fix is not appropriate, can I suggest adding to the documentation to say the data should be encoded with one of these options that seem to work? The Base-64 one is probably better.

PhotoImage(data=data.decode("latin-1).encode("utf-8"))
PhotoImage(data=base64.encodebytes(data))
msg219287 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-28 16:16
All works to me.

>>> import tkinter
>>> b = tkinter.Button()
>>> with open('Lib/test/imghdrdata/python.gif', 'rb') as f: data = f.read()
... 
>>> img = tkinter.PhotoImage(data=data)
>>> b['image'] = img
>>> b.pack()

Could you please provide an example which demonstrates the issue?
msg219317 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-05-28 22:30
Thanks for looking at this. Originally the issue was found by reading the GIF and PNG images on various web pages, such as <http://www.weatherzone.com.au/vic/north-central/castlemaine>. However I was also able to produce the problem with the other formats of that Python logo:

$ python3 -Wall -bt
Python 3.4.1 (default, May 19 2014, 17:40:19) 
[GCC 4.9.0 20140507 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter; tk = tkinter.Tk(); text = tkinter.Text(tk); text.pack()
>>> with open("/lib/python3.4/test/imghdrdata/python.png", "rb") as file: data = file.read()
... 
>>> image = tkinter.PhotoImage(format="png", data=data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/tkinter/__init__.py", line 3384, in __init__
    Image.__init__(self, 'photo', name, cnf, master, **kw)
  File "/usr/lib/python3.4/tkinter/__init__.py", line 3340, in __init__
    self.tk.call(('image', 'create', imgtype, name,) + options)
_tkinter.TclError: CRC check failed
>>> u8png = tkinter.PhotoImage(format="png", data=data.decode("latin-1").encode("utf-8"))
>>> text.image_create(tkinter.END, image=u8png)
'pyimage2'

The same problem occurs with the PGM and PPM logos, and the Base-64 encoding trick does not work with those; only UTF-8 encoding. However as you discovered, the GIF format logo works no problem when passed unencoded, although it continues to work if I use my UTF-8 encoding trick, which is a bit strange. Perhaps the internal UTF-8 decoding step is passing the invalid UTF-8 straight through?
msg219346 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-29 14:27
Here is a patch which fixes passing Python bytes to Tcl in Python 3. However it will be not easy to fix this issue in Python 2.

See also issue21605 which adds tests for Tkinter images (some of them fails without this patch).
msg219499 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-06-01 13:11
Unfortunately we can't use this straightforward and universal solution in Python 2. Here is a patch which adds special workarounds to fix this issue in 2.7.
msg223811 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-07-24 07:04
I will try test the problem and fix on Windows within a day.
msg224295 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-07-30 10:24
The 3.4 patch looks fine, please apply.

I'm -1 on the 2.7 patch. I think it would be better to add a _tkinter helper function to create Tcl byte array objects. Alternatively, the "binary format" command might help. OTOH, I don't care about 2.7, so feel free to do whatever you consider appropriate.
msg224315 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-30 15:34
New changeset 9474f2971855 by Serhiy Storchaka in branch '3.4':
Issue #21580: Now Tkinter correctly handles bytes arguments passed to Tk.
http://hg.python.org/cpython/rev/9474f2971855

New changeset b9d249316f29 by Serhiy Storchaka in branch 'default':
Issue #21580: Now Tkinter correctly handles bytes arguments passed to Tk.
http://hg.python.org/cpython/rev/b9d249316f29
msg224319 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-07-30 15:48
Thank you Martin for your reviews. Here is updated 2.7 patch which implements your suggestion.

The disadvantage of this patch in comparison with first version is that it will not work with statically compiled embedded Python, when only stdlib is updated. I once broke re in bugfix release in such manner (by moving one constant from Python sources to C sources). On other hand, it is very unlikely that anyone uses Tkinter in such circumstances, and in any case this part of code is broken for now, so patch should not introduce new regression.
msg224321 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-07-30 16:09
The 2.7_2 patch looks good to me. I won't rule on the backwards compatibility implications, although I agree that this is unlikely to cause a regression (it would only if somebody updated the standard library only, *and* would use data= for PhotoImage). I'm unsure whether it was possible at all so far to use data= (with whatever argument); if you could have passed a non-buffer object successfully, this would break now.
msg224380 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-31 04:57
New changeset 818989a48e96 by Serhiy Storchaka in branch '2.7':
Issue #21580: Now Tkinter correctly handles binary "data" and "maskdata"
http://hg.python.org/cpython/rev/818989a48e96
History
Date User Action Args
2014-07-31 07:33:13serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-07-31 04:57:13python-devsetmessages: + msg224380
2014-07-30 16:09:38loewissetmessages: + msg224321
2014-07-30 15:48:09serhiy.storchakasetfiles: + tkinter_bytes-2.7_2.patch

messages: + msg224319
2014-07-30 15:34:18python-devsetnosy: + python-dev
messages: + msg224315
2014-07-30 10:24:14loewissetmessages: + msg224295
2014-07-30 07:42:59serhiy.storchakasetnosy: + loewis
2014-07-24 07:04:37terry.reedysetmessages: + msg223811
2014-07-23 15:23:09serhiy.storchakasetnosy: + terry.reedy
2014-06-01 13:11:34serhiy.storchakasetfiles: + tkinter_bytes-2.7.patch

messages: + msg219499
2014-06-01 13:05:54serhiy.storchakasetfiles: + tkinter_bytes.patch
2014-06-01 13:04:51serhiy.storchakasetfiles: - tkinter_bytes.patch
2014-05-29 14:27:25serhiy.storchakasetfiles: + tkinter_bytes.patch

dependencies: + Add tests for Tkinter images
assignee: docs@python -> serhiy.storchaka
versions: + Python 2.7, Python 3.5
keywords: + patch
type: behavior
messages: + msg219346
stage: test needed -> patch review
2014-05-28 22:30:35martin.pantersetmessages: + msg219317
2014-05-28 16:16:43serhiy.storchakasetmessages: + msg219287
stage: test needed
2014-05-26 08:20:56serhiy.storchakasetnosy: + gpolo, serhiy.storchaka
2014-05-26 05:52:24martin.pantercreate