Message267534
Now that #26526 landed (thanks to everybody involved!), I'm requesting a review on an updated version of my patch, which addresses the excessive memory consumption by the parser.
The description of my original patch still applies:
> The attached patch for the parser reduces "Maximum resident set size (kbytes)" threefold, for the degenerate example of 'import ast; ast.parse("0,"*1000000, mode="eval")', by eliminating many CST nodes that have a single child.
>
> According to the comment in Parser/node.c -- "89% of PyObject_REALLOC calls in PyNode_AddChild passed 1 for the size" -- the memory saving should be generally applicable, and not limited just to this degenerate case.
> I've now tried it with "perf.py -r -m", and the memory savings are as follows:
> ...
> on these benchmarks, the saving is not threefold, of course; but still quite substantial (up to 30%).
My new patch updates Modules/parsermodule.c to accept such "compressed" nodes, so that everything still builds cleanly and passes the tests. |
|
Date |
User |
Action |
Args |
2016-06-06 15:09:51 | A. Skrobov | set | recipients:
+ A. Skrobov, fdrake, brett.cannon, rhettinger, paul.moore, vstinner, giampaolo.rodola, christian.heimes, tim.golden, benjamin.peterson, berker.peksag, zach.ware, serhiy.storchaka, eryksun, steve.dower |
2016-06-06 15:09:50 | A. Skrobov | set | messageid: <1465225790.32.0.354602042327.issue26415@psf.upfronthosting.co.za> |
2016-06-06 15:09:50 | A. Skrobov | link | issue26415 messages |
2016-06-06 15:09:50 | A. Skrobov | create | |
|