classification
Title: inefficient pickling of long integers on 64-bit builds
Type: resource usage Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, meador.inge, pitrou, python-dev, rhettinger
Priority: normal Keywords:

Created on 2011-08-12 16:26 by pitrou, last changed 2011-08-13 21:50 by rhettinger. This issue is now closed.

Messages (3)
msg141971 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-08-12 16:26
On a 64-bit Linux machine (where C `long` is 64 bits):

>>> len(pickle.dumps(2**30))
8
>>> len(pickle.dumps(2**31))
16
>>> len(pickle.dumps(2**62))
25
>>> len(pickle.dumps(2**63))
14

This is because the old text protocol is used when the integer can fit in a C long but not in 4 bytes:

>>> pickletools.dis(pickle.dumps(2**62))
    0: \x80 PROTO      3
    2: L    LONG       4611686018427387904
   24: .    STOP
highest protocol among opcodes = 2
>>> pickletools.dis(pickle.dumps(2**63))
    0: \x80 PROTO      3
    2: \x8a LONG1      9223372036854775808
   13: .    STOP
highest protocol among opcodes = 2
msg142032 - (view) Author: Roundup Robot (python-dev) Date: 2011-08-13 18:19
New changeset 8e824e09924a by Antoine Pitrou in branch 'default':
Issue #12744: Fix inefficient representation of integers
http://hg.python.org/cpython/rev/8e824e09924a
msg142040 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-08-13 21:50
Nice.
History
Date User Action Args
2011-08-13 21:50:57rhettingersetnosy: + rhettinger
messages: + msg142040
2011-08-13 18:19:54pitrousetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2011-08-13 18:19:40python-devsetnosy: + python-dev
messages: + msg142032
2011-08-13 16:41:58pitrousetnosy: + alexandre.vassalotti
2011-08-12 16:39:08meador.ingesetstage: needs patch
2011-08-12 16:28:48meador.ingesetnosy: + meador.inge
2011-08-12 16:26:23pitroucreate