This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: sys.getsizeof wrong for Py3k bool objects
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.0, Python 3.1
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Alexander.Belopolsky, loewis, mark.dickinson, schuppenies
Priority: normal Keywords: patch

Created on 2008-08-26 20:52 by schuppenies, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
bool_sizeof.patch schuppenies, 2008-08-26 20:52 Patch against py3k branch, revision 66040
smallints_sizeof.patch schuppenies, 2008-09-23 13:19
Messages (9)
msg71996 - (view) Author: Robert Schuppenies (schuppenies) * (Python committer) Date: 2008-08-26 20:52
sys.getsizeof returns wrong results for bool objects in Python 3000.
Although bool objects use the same datatype as long objects, they are
allocated differently. Thus, the inherited long_sizeof implementation is
incorrect. The applied patch addresses this issue.
msg72742 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-09-07 16:45
I'm not sure this is a bug. sys.getsizeof doesn't take padding in the
malloc implementation into account, either, so a long object that
accounts to 22 bytes (such as the number 1) uses at least 24 bytes,
also. In any case, I also think this doesn't matter much either way.
msg73208 - (view) Author: Robert Schuppenies (schuppenies) * (Python committer) Date: 2008-09-14 10:12
As I understood the long object allocation it is implemented as
"PyObject_MALLOC(sizeof(PyVarObject) + size*sizeof(digit))" to avoid
this allocation of extra 2 bytes. So from my understanding, the number 0
allocates memory for the reference count, type, and ob_size, whereas any
other number allocates this plus additional memory required by the
number of digits.

Looking at bool objects in Py3k, arn't they fixed-sized memory-wise,
always allocating the the padded size of _longobject?

> In any case, I also think this doesn't matter much either way. 
Why do you think so?
msg73228 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-09-14 16:23
>> In any case, I also think this doesn't matter much either way. 
> Why do you think so?

What's the actual difference that this change makes? At most 8
bytes per object, right? And for two objects in total. So if somebody
would compute memory consumption, they might be off by not more
than 14 bytes, in total. Compared to all the other errors that memory
computation makes (e.g. malloc headers, rounding-up to multiples of
8 in obmalloc) which aren't accounted-for in sys.getsizeof, this
difference is negligible.

What's more, the small_ints aren't dynamically allocated, either,
but instead, each small_int takes a complete PyLongObject. If
that was also considered in long_sizeof, the computation would happen
to be completely correct for bool also.
msg73250 - (view) Author: Robert Schuppenies (schuppenies) * (Python committer) Date: 2008-09-15 08:12
> What's the actual difference that this change makes?

It would provide more accurate results, even in the light of being not
perfect.

> [..] each small_int takes a complete PyLongObject. If that was also
> considered in long_sizeof, the computation would happen to be
> completely correct for bool also.

So how should this bug report be handled? Provide a patch to handle
getsizeof correctly for small_ints? 'wont fix' because there are issues
anyway? I would prefer the former and try to come up with a patch if you
think it is worthwhile.
msg73252 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-09-15 09:44
> So how should this bug report be handled? Provide a patch to handle
> getsizeof correctly for small_ints? 'wont fix' because there are issues
> anyway? I would prefer the former and try to come up with a patch if you
> think it is worthwhile.

Fixing it for small_ints would be fine with me - there is specialized
code for long sizeof already. It's the explosion of boolobject that I
dislike.
msg73634 - (view) Author: Robert Schuppenies (schuppenies) * (Python committer) Date: 2008-09-23 13:19
Attached is a patch which takes the preallocation of small_ints into
account. What do you think?
msg101310 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-03-19 08:36
I don't think there's anything worth fixing here.  It's true that getsizeof is sometimes going to return results that are too small, because there are a good few places in the longobject internals where it's not predictable in advance exactly how much space is needed, so memory is overallocated.

The case of the small int 0 is one example of this, but it's far from the only one.  For example, if you multiply a 2-limb long by another 2-limb long the code will always allocate 4 limbs for the result, even though it'll often turn out that the result fits in 3 limbs.  Should sys.getsizeof return base_size + 4 * sizeof_limb in that case, instead of base_size + 3 * sizeof_limb?  That would be difficult to achieve, since long objects don't currently know how much space was actually allocated to hold them.
msg101313 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-03-19 09:20
Closing this as "won't fix", then.
History
Date User Action Args
2022-04-11 14:56:38adminsetgithub: 47940
2010-03-19 09:20:41loewissetstatus: open -> closed
resolution: wont fix
messages: + msg101313
2010-03-19 08:36:28mark.dickinsonsetnosy: + mark.dickinson
messages: + msg101310
2010-03-19 05:24:29Alexander.Belopolskysetnosy: + Alexander.Belopolsky
2008-09-23 13:19:04schuppeniessetfiles: + smallints_sizeof.patch
messages: + msg73634
2008-09-15 09:44:29loewissetmessages: + msg73252
2008-09-15 08:12:01schuppeniessetmessages: + msg73250
2008-09-14 16:23:01loewissetmessages: + msg73228
2008-09-14 10:12:25schuppeniessetmessages: + msg73208
2008-09-07 16:45:02loewissetnosy: + loewis
messages: + msg72742
2008-08-26 20:52:02schuppeniescreate