This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author lemburg
Recipients amaury.forgeotdarc, jafo, lemburg, orivej, pitrou
Date 2008-03-20.22:13:01
SpamBayes Score 0.09247112
Marked as misclassified No
Message-id <1206051185.04.0.300326312593.issue1943@psf.upfronthosting.co.za>
In-reply-to
Content
Thanks for running the tests again. The use of pymalloc for the buffer 
made a significant difference indeed. I expect that more can be had by
additionally tweaking KEEPALIVE_SIZE_LIMIT.

It is interesting to see that the free list patch only appears to
provide better timings for the "smaller" strings tests. I couldn't find
the INTBENCH file anywhere in the Python source tree, but here's the
distribution of the words of the latter two files:

Dev-Python/LICENSE:
------------------------------------------------------------------------
Length: [Count] 0----1----2----3----4----5----6----7----8----9----10 * 10%
     1: [   28] ===
     2: [  350] =================================================
     3: [  354] ==================================================
     4: [  190] ==========================
     5: [  191] ==========================
     6: [  158] ======================
     7: [  164] =======================
     8: [  132] ==================
     9: [  127] =================
    10: [  102] ==============
    11: [   39] =====
    12: [   34] ====
    13: [   21] ==
    14: [   10] =
    15: [   10] =
    16: [    0]
    17: [    0]
    18: [    1]
    19: [    0]
    20: [    0]
    21: [    1]
    22: [    0]
    23: [    0]
    24: [    0]
    25: [    1]
    26: [    1]
    27: [    1]
    28: [    0]
    29: [    1]
    30: [    0]
    31: [    0]
    32: [    0]
    33: [    0]
    34: [    0]
    35: [    0]
    36: [    2]
    37: [    0]
    38: [    0]
    39: [    1]
    40: [    0]
    41: [    0]
    42: [    0]
    43: [    1]
    44: [    1]
    45: [    0]
    46: [    0]
    47: [    0]
    48: [    0]
    49: [    0]
    50: [    1]
    51: [    0]
    52: [    0]
    53: [    0]
    54: [    0]
    55: [    0]
    56: [    0]
    57: [    0]
    58: [    0]
    59: [    0]
    60: [    0]
    61: [    0]
    62: [    0]
    63: [    1]

Dev-Python/Misc/HISTORY:
------------------------------------------------------------------------
Length: [Count] 0----1----2----3----4----5----6----7----8----9----10 * 10%
     1: [ 6853] ==================
     2: [13920] =====================================
     3: [18401] ==================================================
     4: [12626] ==================================
     5: [ 9545] =========================
     6: [ 9348] =========================
     7: [ 9625] ==========================
     8: [ 7351] ===================
     9: [ 5353] ==============
    10: [ 3266] ========
    11: [ 1947] =====
    12: [ 1336] ===
    13: [  983] ==
    14: [  638] =
    15: [  408] =
    16: [  288]
    17: [  286]
    18: [  216]
    19: [  176]
    20: [  147]
    21: [  120]
    22: [  116]
    23: [   85]
    24: [   89]
    25: [   70]
    26: [   44]
    27: [   59]
    28: [   39]
    29: [   32]
    30: [   65]
    31: [   23]
    32: [   26]
    33: [   28]
    34: [   19]
    35: [   18]
    36: [    9]
    37: [   18]
    38: [    5]
    39: [   10]
    40: [    9]
    41: [    9]
    42: [    1]
    43: [    1]
    44: [    8]
    45: [    4]
    46: [    5]
    47: [    5]
    48: [    3]
    49: [    3]
    50: [    4]
    51: [    0]
    52: [    0]
    53: [    2]
    54: [    0]
    55: [    0]
    56: [    4]
    57: [    2]
    58: [    4]
    59: [    1]
    60: [    0]
    61: [    1]
    62: [    0]
    63: [    1]
    64: [    1]
    65: [    9]
    66: [    1]
    67: [    1]
    68: [    0]
    69: [    0]
    70: [   16]
    71: [    1]
    72: [    0]
    73: [    1]

Compare that to a typical Python module source file...

Dev-Python/Lib/urllib.py:
------------------------------------------------------------------------
Length: [Count] 0----1----2----3----4----5----6----7----8----9----10 * 10%
     1: [  806] ==================================================
     2: [  672] =========================================
     3: [  675] =========================================
     4: [  570] ===================================
     5: [  531] ================================
     6: [  501] ===============================
     7: [  296] ==================
     8: [  393] ========================
     9: [  246] ===============
    10: [  147] =========
    11: [  150] =========
    12: [  102] ======
    13: [   90] =====
    14: [  116] =======
    15: [   61] ===
    16: [   51] ===
    17: [   45] ==
    18: [   38] ==
    19: [   31] =
    20: [   39] ==
    21: [   24] =
    22: [   18] =
    23: [   18] =
    24: [   23] =
    25: [   17] =
    26: [   15]
    27: [   13]
    28: [   14]
    29: [   11]
    30: [    9]
    31: [    7]
    32: [    1]
    33: [    6]
    34: [   10]
    35: [    2]
    36: [    4]
    37: [    3]
    38: [    6]
    39: [    1]
    40: [    0]
    41: [    1]
    42: [    5]
    43: [    0]
    44: [    0]
    45: [    0]
    46: [    0]
    47: [    0]
    48: [    2]
    49: [    0]
    50: [    1]
    51: [    1]
    52: [    2]

The distributions differ a lot, but they both show that typical strings
(both in English text and Python program code) have a length of < 25
characters.

Setting KEEPALIVE_SIZE_LIMIT to 32 should cover most of those cases
while still keeping the memory load within bounds. Raising the free list
limit to e.g. 2048 would cause at most 64kB to be used by the free list
- which is well within bounds of any modern CPU L2 cache.
History
Date User Action Args
2008-03-20 22:13:05lemburgsetspambayes_score: 0.0924711 -> 0.09247112
recipients: + lemburg, jafo, amaury.forgeotdarc, pitrou, orivej
2008-03-20 22:13:05lemburgsetspambayes_score: 0.0924711 -> 0.0924711
messageid: <1206051185.04.0.300326312593.issue1943@psf.upfronthosting.co.za>
2008-03-20 22:13:04lemburglinkissue1943 messages
2008-03-20 22:13:01lemburgcreate