classification
Title: __sizeof__ of array should include size of items
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: meador.inge Nosy List: Pankrat, georg.brandl, jcea, loewis, meador.inge, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2012-07-22 16:22 by Pankrat, last changed 2012-08-11 04:31 by meador.inge. This issue is now closed.

Files
File name Uploaded Description Edit
array_sizeof_3.3.patch Pankrat, 2012-08-07 09:47 Implement __sizeof__() for array (Python 3.3) review
array_sizeof_2.7.patch Pankrat, 2012-08-10 12:24 Implement __sizeof__() for array (Python 2.7) review
array_sizeof_3.2.patch Pankrat, 2012-08-10 12:24 Implement __sizeof__() for array (Python 3.2) review
Messages (18)
msg166147 - (view) Author: Ludwig Hähne (Pankrat) * Date: 2012-07-22 16:22
If sys.getsizeof is called on an array, the result doesn't include the size of the items: 

>>> from array import array
>>> a = array("i", [0] * 100000)
>>> a.__sizeof__()
40

While this makes sense for a list, an array doesn't have separate referents that could be sized:

>>> import gc
>>> gc.get_referents(a)
[]

The attached patch adds an implementation of the __sizeof__ method for arrays that includes the size of the buffer for the elements.

It would be great if the patch would be considered for one of the upcoming versions of Python.

Thanks, Ludwig
msg166150 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-07-22 16:58
Can you please submit a contributor form?
msg166153 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-07-22 17:25
I think it should be applied to 3.2 too.
msg166155 - (view) Author: Ludwig Hähne (Pankrat) * Date: 2012-07-22 18:34
Uses self->allocated instead of Py_SIZE as Martin suggested.
msg166157 - (view) Author: Ludwig Hähne (Pankrat) * Date: 2012-07-22 18:57
Yes, it would be great if the change could be applied to 3.2 (and 2.7) but I guess not many people will be affected by the problem.

BTW, I just submitted the contributor form.
msg166221 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012-07-23 13:27
With the precedent of issue #15402, I think that 2.7 and 3.2 should be fine.
msg166388 - (view) Author: Ludwig Hähne (Pankrat) * Date: 2012-07-25 11:55
Revised the patch based on the discussion in issue #15402 (http://bugs.python.org/msg166372). It doesn't typecast the function anymore and instead casts to the array type inside the function.
msg167487 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-08-05 11:19
Ludwig, you can use some helpers from test.support (see issue15467) for __sizeof__ testing.
msg167590 - (view) Author: Ludwig Hähne (Pankrat) * Date: 2012-08-06 23:11
Serhiy, the tests now explicitly check the base size of the array.

I'm not sure if I got the base size compution right (only checked on 32bit Linux). Could you check that the struct definition in the test makes sense and matches the C arrayobject definition? Thanks, Ludwig
msg167596 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2012-08-07 01:49
I left some commits in Rietveld.  Otherwise, looks OK.
msg167610 - (view) Author: Ludwig Hähne (Pankrat) * Date: 2012-08-07 09:47
Meador, thanks for reviewing. The updated patch is now attached to the bug.
msg167616 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-08-07 10:21
Patch looks good to me, however tests for 3.2 and 2.7 should be modified (change "n" struct format specifier to "P", remove last "i" and use test_support instead test.support (or import test_support as support) in 2.7).
msg167649 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2012-08-08 01:49
Georg, yet another __sizeof__ fix.  OK for 3.3?
msg167666 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-08-08 06:09
LGTM.
msg167866 - (view) Author: Ludwig Hähne (Pankrat) * Date: 2012-08-10 12:28
Meador, Serhiy, I've add patches for Python 2.7 and Python 3.2 following Serhiy suggestions.
msg167889 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-08-10 16:33
Patches look good to me.
msg167932 - (view) Author: Roundup Robot (python-dev) Date: 2012-08-11 04:26
New changeset 91382d4e2dfb by Meador Inge in branch '2.7':
Issue #15424: Add a __sizeof__ implementation for array objects.
http://hg.python.org/cpython/rev/91382d4e2dfb

New changeset 45ef9bc8739f by Meador Inge in branch '3.2':
Issue #15424: Add a __sizeof__ implementation for array objects.
http://hg.python.org/cpython/rev/45ef9bc8739f

New changeset 7d82a60850fd by Meador Inge in branch 'default':
Issue #15424: Add a __sizeof__ implementation for array objects.
http://hg.python.org/cpython/rev/7d82a60850fd
msg167934 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2012-08-11 04:31
Thanks for the patches!  (I fixed a minor nit on the 2.7 patch where 'array_sizeof' was defined inside #ifdef Py_USING_UNICODE).
History
Date User Action Args
2012-08-11 04:31:26meador.ingesetstatus: open -> closed
resolution: fixed
messages: + msg167934

stage: commit review -> resolved
2012-08-11 04:26:51python-devsetnosy: + python-dev
messages: + msg167932
2012-08-10 16:33:00serhiy.storchakasetmessages: + msg167889
2012-08-10 12:28:40Pankratsetmessages: + msg167866
2012-08-10 12:25:48Pankratsetfiles: - array_sizeof.patch
2012-08-10 12:25:41Pankratsetfiles: - array_sizeof_v2.patch
2012-08-10 12:25:34Pankratsetfiles: - array_sizeof_v3.patch
2012-08-10 12:25:24Pankratsetfiles: - array_sizeof_v4.patch
2012-08-10 12:24:39Pankratsetfiles: + array_sizeof_3.2.patch
2012-08-10 12:24:19Pankratsetfiles: + array_sizeof_2.7.patch
2012-08-08 06:09:18georg.brandlsetmessages: + msg167666
2012-08-08 01:49:51meador.ingesetnosy: + georg.brandl

messages: + msg167649
stage: commit review
2012-08-07 10:21:23serhiy.storchakasetmessages: + msg167616
2012-08-07 09:47:06Pankratsetfiles: + array_sizeof_3.3.patch

messages: + msg167610
2012-08-07 01:49:31meador.ingesetassignee: meador.inge
messages: + msg167596
2012-08-06 23:11:17Pankratsetfiles: + array_sizeof_v4.patch

messages: + msg167590
2012-08-05 11:19:30serhiy.storchakasetmessages: + msg167487
2012-07-25 11:55:44Pankratsetfiles: + array_sizeof_v3.patch

messages: + msg166388
2012-07-23 15:44:38jceasetnosy: + meador.inge

versions: + Python 3.2, - Python 3.4
2012-07-23 13:27:19jceasetnosy: + jcea
messages: + msg166221
2012-07-22 18:57:19Pankratsetmessages: + msg166157
2012-07-22 18:34:25Pankratsetfiles: + array_sizeof_v2.patch

messages: + msg166155
2012-07-22 17:25:17serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg166153
2012-07-22 16:58:10loewissetnosy: + loewis
messages: + msg166150
2012-07-22 16:22:41Pankratcreate