Author A. Skrobov
Recipients A. Skrobov, benjamin.peterson, berker.peksag, brett.cannon, christian.heimes, eryksun, fdrake, giampaolo.rodola, paul.moore, rhettinger, serhiy.storchaka, steve.dower, tim.golden, vstinner, zach.ware
Date 2016-06-06.15:09:49
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1465225790.32.0.354602042327.issue26415@psf.upfronthosting.co.za>
In-reply-to
Content
Now that #26526 landed (thanks to everybody involved!), I'm requesting a review on an updated version of my patch, which addresses the excessive memory consumption by the parser.

The description of my original patch still applies:

> The attached patch for the parser reduces "Maximum resident set size (kbytes)" threefold, for the degenerate example of 'import ast; ast.parse("0,"*1000000, mode="eval")', by eliminating many CST nodes that have a single child.
>
> According to the comment in Parser/node.c -- "89% of PyObject_REALLOC calls in PyNode_AddChild passed 1 for the size" -- the memory saving should be generally applicable, and not limited just to this degenerate case.

> I've now tried it with "perf.py -r -m", and the memory savings are as follows:
> ...
> on these benchmarks, the saving is not threefold, of course; but still quite substantial (up to 30%).

My new patch updates Modules/parsermodule.c to accept such "compressed" nodes, so that everything still builds cleanly and passes the tests.
History
Date User Action Args
2016-06-06 15:09:51A. Skrobovsetrecipients: + A. Skrobov, fdrake, brett.cannon, rhettinger, paul.moore, vstinner, giampaolo.rodola, christian.heimes, tim.golden, benjamin.peterson, berker.peksag, zach.ware, serhiy.storchaka, eryksun, steve.dower
2016-06-06 15:09:50A. Skrobovsetmessageid: <1465225790.32.0.354602042327.issue26415@psf.upfronthosting.co.za>
2016-06-06 15:09:50A. Skrobovlinkissue26415 messages
2016-06-06 15:09:50A. Skrobovcreate