classification
Title: Tkinter doesn't support large integers
Type: enhancement Stage: resolved
Components: Tkinter Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: benjamin.peterson, gpolo, haypo, loewis, python-dev, serhiy.storchaka
Priority: normal Keywords: needs review, patch

Created on 2013-01-02 15:09 by serhiy.storchaka, last changed 2015-04-22 12:33 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
tkinter_bignum_4.patch serhiy.storchaka, 2015-02-21 17:40 review
tkinter_hexversion.patch serhiy.storchaka, 2015-04-21 08:49 review
Messages (21)
msg178815 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-02 15:09
A long time Tcl got support first 64-bit integers, and then arbitrary size integers. Python also supports arbitrary size integers. However Tkinter supports only C long integers. For example, on 32-bit platform:

>>> import tkinter
>>> t = tkinter.Tk()
>>> t.tk.call('expr', 2**30)
1073741824
>>> t.tk.call('expr', 2**31)
<wideInt object at 0x9122518>
>>> t.tk.call('expr', 2**63)
<bignum object at 0x9126010>

Those <wideInt object> and <bignum object> are not usable from Python. Potentially this can cause errors in some rare circumstances.

I'm working on a patch, but was faced with a problem. Tcl provides functions for conversions between Bignum Tcl object and mp_int, but it does not provide all functions for conversions between mp_int and bytes sequence. Which is better, add libtommath library to CPython dependencies, or implement the missing functionality manually? Or may be use haxadecimal representation for conversions?
msg178988 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-03 20:49
Here is a patch which adds support of "wideInt" and "bignum" Tcl types.
msg179204 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-06 18:40
I reclassify this as enhancement until the real bug is not found.
msg195916 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-08-22 19:27
Patch updated.
msg218743 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-18 13:48
Patch synchronized with tip and fixed support of Tcl <8.5.
msg235098 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-31 10:15
Here is test failure related to this issue:

http://buildbot.python.org/all/builders/AMD64%20Windows8%202.7/builds/156/steps/test/logs/stdio

test test_tk failed -- Traceback (most recent call last):
  File "D:\buildarea\2.7.bolen-windows8\build\lib\lib-tk\test\test_tkinter\test_widgets.py", line 91, in test_use
    wid = parent.winfo_id()
  File "D:\buildarea\2.7.bolen-windows8\build\lib\lib-tk\Tkinter.py", line 839, in winfo_id
    self.tk.call('winfo', 'id', self._w))
TclError: integer value too large to represent
msg236375 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-21 17:40
Updated patch now supports wideInt if PY_LONG_LONG is not defined or is not equal to Tcl_WideInt. getint() also now supports big integers (currently it returns ambiguous result for values outside the range of C signed int).

As far as this issue causes random tests failures, may be it makes sense to apply the patch to maintained releases. On other hand, looks as this happens very rarely.
msg236381 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2015-02-21 18:06
I'm all for fixing random test failures.
msg236386 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-21 19:02
But there is a risk to break Python builds with old Tcl/Tk. We dropped support of Tcl/Tk older than 8.4 in 3.5, but theoretically 2.7 should work with older versions. It was not tested for years, we have no build bots with 8.3, and I manually test Python only with 8.4+ (it is harder to build 8.3, not mentioning older versions). But AFAIK wideInt was not supported in 8.3, so code and tests should be adapted for optional support of integers outside 32-bit range.

I could try to build 8.3 and adapt the patch, but I'm not very motivated in this. And I can't take the responsibility of committing the patch blindly, without testing with different options.
msg236387 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2015-02-21 19:19
It's up to you. Since 2.7 is so long lived, we've had to gradually alter supported library versions (e.g. for bsddb).
msg239928 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-02 15:49
New changeset 42a6449e577c by Serhiy Storchaka in branch '2.7':
Issue #16840: Tkinter now supports 64-bit integers added in Tcl 8.4 and
https://hg.python.org/cpython/rev/42a6449e577c

New changeset a6f4d8fa7ab8 by Serhiy Storchaka in branch '3.4':
Issue #16840: Tkinter now supports 64-bit integers added in Tcl 8.4 and
https://hg.python.org/cpython/rev/a6f4d8fa7ab8

New changeset 9291b28157e1 by Serhiy Storchaka in branch 'default':
Issue #16840: Tkinter now supports 64-bit integers added in Tcl 8.4 and
https://hg.python.org/cpython/rev/9291b28157e1
msg239932 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-02 17:08
New changeset 9905fb0b5885 by Serhiy Storchaka in branch '2.7':
Issue #16840: Fixed test_tcl for Tcl < 8.5.
https://hg.python.org/cpython/rev/9905fb0b5885

New changeset 1d2444273b3d by Serhiy Storchaka in branch '3.4':
Issue #16840: Fixed test_tcl for Tcl < 8.5.
https://hg.python.org/cpython/rev/1d2444273b3d

New changeset 2398dba039f3 by Serhiy Storchaka in branch 'default':
Issue #16840: Fixed test_tcl for Tcl < 8.5.
https://hg.python.org/cpython/rev/2398dba039f3
msg240224 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2015-04-07 19:31
test_expr_bignum() should tolerate the long type:

http://buildbot.python.org/all/builders/x86%20Tiger%202.7/builds/3007/steps/test/logs/stdio

======================================================================
FAIL: test_expr_bignum (test.test_tcl.TclTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/db3l/buildarea/2.7.bolen-tiger/build/Lib/test/test_tcl.py", line 443, in test_expr_bignum
    self.assertIsInstance(result, type(int(result)))
AssertionError: -2147483648L is not an instance of <type 'int'>

In fact, I don't understand the test self.assertIsInstance(result, type(int(result))). What do you expect from type(int(x)): int or long depending on the value of x? The problem is that an intermediate result may be a long and so tcl.call('expr', ..) returns a long even if it may fit into a small int.

I suggest to simply drop the test self.assertIsInstance(result, type(int(result))).
msg240228 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-07 20:20
It is important that the result is an int at least for small ints. So I prefer to keep limited test.

-            self.assertIsInstance(result, type(int(result)))
+            if abs(result) < 2**31:
+                self.assertIsInstance(result, int)
msg240442 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-10 18:13
New changeset 350c78a92046 by Serhiy Storchaka in branch '2.7':
Issue #16840: Fixed Tcl test on 2.7 with Tcl 8.4.19.
https://hg.python.org/cpython/rev/350c78a92046
msg241628 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-20 10:05
There is at least one buildbot that now fails to build _tkinter.

http://buildbot.python.org/all/builders/x86%20FreeBSD%206.4%202.7/builds/2457/steps/test/logs/stdio
building '_tkinter' extension
gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -g -O0 -Wall -Wstrict-prototypes -DWITH_APPINIT=1 -I/usr/local/include/tcl8.5 -I/usr/local/include/tk8.5 -I/usr/X11R6/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/home/db3l/buildarea/2.7.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/2.7.bolen-freebsd/build -c /usr/home/db3l/buildarea/2.7.bolen-freebsd/build/Modules/_tkinter.c -o build/temp.freebsd-6.4-RELEASE-i386-2.7-pydebug/usr/home/db3l/buildarea/2.7.bolen-freebsd/build/Modules/_tkinter.o
/usr/home/db3l/buildarea/2.7.bolen-freebsd/build/Modules/_tkinter.c:101:24: tclTomMath.h: No such file or directory
msg241630 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-20 11:08
New changeset 8ab077c22fbf by Serhiy Storchaka in branch '2.7':
Issue #16840: Turn on support of bignums only in final release of Tcl 8.5.
https://hg.python.org/cpython/rev/8ab077c22fbf

New changeset 7f1622478d17 by Serhiy Storchaka in branch '3.4':
Issue #16840: Turn on support of bignums only in final release of Tcl 8.5.
https://hg.python.org/cpython/rev/7f1622478d17

New changeset 7a7f09528866 by Serhiy Storchaka in branch 'default':
Issue #16840: Turn on support of bignums only in final release of Tcl 8.5.
https://hg.python.org/cpython/rev/7a7f09528866
msg241706 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-21 07:31
This didn't help. Actually we should turn on bignum support only since 8.5.8, because tclTomMath.h was broken before. But this is not related, because tclTomMath.h is not found itself on this buildbot.
msg241708 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-21 08:47
Ah, we should also test full version of 8.6, because bignums was not supported in earlier alphas and betas (if I remember correctly, this buildbot uses beta version of 8.6).

TK_VERSION_HEX packs fields in wrong order, not suitable for comparing of non-final releases. It's value for 8.6a3 is larger than 8.6.1.

Here is a patch that adds TK_HEX_VERSION with correctly packed fields and turn off bignum support in non-final 8.6.
msg241729 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-21 18:18
New changeset 5116916e4f7f by Serhiy Storchaka in branch '2.7':
Issue #16840. Turn off bignum support in tkinter with with Tcl earlier than 8.5.8
https://hg.python.org/cpython/rev/5116916e4f7f
msg241785 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-22 08:02
New changeset 4bf210f59ac6 by Serhiy Storchaka in branch '2.7':
Issue #16840: Skip bignum tests on minor releases where they are not supported.
https://hg.python.org/cpython/rev/4bf210f59ac6

New changeset 97519d85b5c8 by Serhiy Storchaka in branch '3.4':
Issue #16840. Turn off bignum support in tkinter with with Tcl earlier than 8.5.8
https://hg.python.org/cpython/rev/97519d85b5c8

New changeset 701b830abbf0 by Serhiy Storchaka in branch 'default':
Issue #16840. Turn off bignum support in tkinter with with Tcl earlier than 8.5.8
https://hg.python.org/cpython/rev/701b830abbf0
History
Date User Action Args
2015-04-22 12:33:08serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-04-22 08:02:38python-devsetmessages: + msg241785
2015-04-21 18:18:04python-devsetmessages: + msg241729
2015-04-21 08:49:10serhiy.storchakasetfiles: + tkinter_hexversion.patch
stage: resolved -> patch review
resolution: fixed -> (no value)
versions: + Python 2.7, Python 3.4
2015-04-21 08:47:53serhiy.storchakasetmessages: + msg241708
2015-04-21 07:31:39serhiy.storchakasetmessages: + msg241706
2015-04-20 11:08:15python-devsetmessages: + msg241630
2015-04-20 10:05:58serhiy.storchakasetstatus: closed -> open

messages: + msg241628
2015-04-10 18:14:54serhiy.storchakasetstatus: open -> closed
resolution: fixed
2015-04-10 18:13:50python-devsetmessages: + msg240442
2015-04-07 20:20:50serhiy.storchakasetmessages: + msg240228
2015-04-07 19:31:45hayposetstatus: closed -> open

nosy: + haypo
messages: + msg240224

resolution: fixed -> (no value)
2015-04-02 22:28:11serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-04-02 17:08:11python-devsetmessages: + msg239932
2015-04-02 15:49:46python-devsetnosy: + python-dev
messages: + msg239928
2015-02-21 19:19:49benjamin.petersonsetmessages: + msg236387
2015-02-21 19:02:55serhiy.storchakasetmessages: + msg236386
2015-02-21 18:06:40benjamin.petersonsetmessages: + msg236381
2015-02-21 17:41:51serhiy.storchakasetfiles: - tkinter_bignum_3.patch
2015-02-21 17:41:13serhiy.storchakasetfiles: - tkinter_bignum_2.patch
2015-02-21 17:41:06serhiy.storchakasetfiles: - tkinter_bignum.patch
2015-02-21 17:40:50serhiy.storchakasetfiles: + tkinter_bignum_4.patch

nosy: + benjamin.peterson
messages: + msg236375

keywords: + needs review
2015-01-31 10:15:40serhiy.storchakasetmessages: + msg235098
versions: + Python 3.5, - Python 3.4
2014-05-18 16:07:27serhiy.storchakasetnosy: + loewis
2014-05-18 13:48:56serhiy.storchakasetfiles: + tkinter_bignum_3.patch

messages: + msg218743
2013-08-22 19:27:10serhiy.storchakasetfiles: + tkinter_bignum_2.patch

messages: + msg195916
2013-01-06 18:40:04serhiy.storchakasettype: behavior -> enhancement
messages: + msg179204
versions: - Python 2.7, Python 3.2, Python 3.3
2013-01-03 21:53:17serhiy.storchakasetassignee: serhiy.storchaka
2013-01-03 20:49:39serhiy.storchakasetfiles: + tkinter_bignum.patch
messages: + msg178988

assignee: serhiy.storchaka -> (no value)
keywords: + patch
stage: needs patch -> patch review
2013-01-02 15:09:13serhiy.storchakacreate