Author kristjan.jonsson
Recipients christian.heimes, gregory.p.smith, kristjan.jonsson, loewis, mark.dickinson, pitrou, serhiy.storchaka
Date 2012-11-20.10:14:55
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1353406496.97.0.397769351442.issue16475@psf.upfronthosting.co.za>
In-reply-to
Content
Serhiy, to answer your questions:

> Can you please measure the time of unmarshalling?
Sure. Normally, I wouldn't consider time to be important here because for code objects, unmarshaling is done only once per run  However, testing it is simple:
I added timeit(number=100) for unmarshaling of the different version code.
0.09614 0.09605 0.582672 0.553599
0.05590 0.05628 0.293370 0.285517
0.05703 0.05754 0.294601 0.292070
The lines are version 2, 3, and 4(*) respectively.  The columns are test_marshal.py, recode.intern(test_marshal.py), decimal.py and recode.intern(decimal.py)

As you see, loading time is almost halfed with version 3 and 4 compared to 2.  Version 3 is also slightly faster than 4
(*)verion 4 is a "special" version that only instances strings.


> If you can count the statistics about marshalled types
> (what percent of shared and non shared integers, strings, etc),
> it would also be very interesting.
That's more tricky, at least on a type=by-type basis, but I could do a global object count.  Later.

> There is at least one reason. This increases size of the refs table.
I checked this, by printing out the size of the instance list when loads() was done:
457 1571
297 1163
429 1539
The columns are test_marshal.py and decimal.py
the lines are version 3, version 4( only strings ) and special version5 which is like 3 but omits ints.
As you see, the ints correspond to roughly 6% and 2% of the instances respectively.  The bulk of the list is taken up by strings (65% and 74%)

This shows that adding instancing of all other types on top of the strings does not typically expand the instance list more than 50%
History
Date User Action Args
2012-11-20 10:14:57kristjan.jonssonsetrecipients: + kristjan.jonsson, loewis, gregory.p.smith, mark.dickinson, pitrou, christian.heimes, serhiy.storchaka
2012-11-20 10:14:56kristjan.jonssonsetmessageid: <1353406496.97.0.397769351442.issue16475@psf.upfronthosting.co.za>
2012-11-20 10:14:56kristjan.jonssonlinkissue16475 messages
2012-11-20 10:14:55kristjan.jonssoncreate