This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author serhiy.storchaka
Recipients methane, rhettinger, serhiy.storchaka
Date 2017-04-01.11:52:41
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1491047561.92.0.226119974337.issue29961@psf.upfronthosting.co.za>
In-reply-to
Content
For now new set and frozenset objects can allocate 2 times larger table than necessary when are created from set or dict. For example if the size n of the original set is the power of two, resulting set will allocate the table of 4*n rather that 2*n. Up to 20% of new sets use twice more memory than necessary.

Proposed patch makes set_merge() allocating the table of size n*5/3 instead of n*2. This is the minimal size necessary for inserting all elements with fill ration <=60%.

$ ./python -c 'N = 6000; from sys import getsizeof; s = [getsizeof(frozenset(set(range(n)))) for n in range(N)]; print( [(n, s[n]) for n in range(N) if not n or s[n] != s[n-1]] )'
Unpatched:  [(0, 112), (5, 240), (8, 368), (16, 624), (32, 1136), (64, 2160), (128, 4208), (256, 8304), (512, 16496), (1024, 32880), (2048, 65648), (4096, 131184)]
Patched:    [(0, 112), (5, 240), (9, 368), (19, 624), (38, 1136), (77, 2160), (153, 4208), (307, 8304), (614, 16496), (1229, 32880), (2457, 65648), (4915, 131184)]
History
Date User Action Args
2017-04-01 11:52:41serhiy.storchakasetrecipients: + serhiy.storchaka, rhettinger, methane
2017-04-01 11:52:41serhiy.storchakasetmessageid: <1491047561.92.0.226119974337.issue29961@psf.upfronthosting.co.za>
2017-04-01 11:52:41serhiy.storchakalinkissue29961 messages
2017-04-01 11:52:41serhiy.storchakacreate