This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author A. Skrobov
Recipients A. Skrobov, benjamin.peterson, berker.peksag, brett.cannon, christian.heimes, eryksun, fdrake, giampaolo.rodola, paul.moore, rhettinger, serhiy.storchaka, steve.dower, tim.golden, vstinner, zach.ware
Date 2016-06-06.15:09:49
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1465225790.32.0.354602042327.issue26415@psf.upfronthosting.co.za>
In-reply-to
Content
Now that #26526 landed (thanks to everybody involved!), I'm requesting a review on an updated version of my patch, which addresses the excessive memory consumption by the parser.

The description of my original patch still applies:

> The attached patch for the parser reduces "Maximum resident set size (kbytes)" threefold, for the degenerate example of 'import ast; ast.parse("0,"*1000000, mode="eval")', by eliminating many CST nodes that have a single child.
>
> According to the comment in Parser/node.c -- "89% of PyObject_REALLOC calls in PyNode_AddChild passed 1 for the size" -- the memory saving should be generally applicable, and not limited just to this degenerate case.

> I've now tried it with "perf.py -r -m", and the memory savings are as follows:
> ...
> on these benchmarks, the saving is not threefold, of course; but still quite substantial (up to 30%).

My new patch updates Modules/parsermodule.c to accept such "compressed" nodes, so that everything still builds cleanly and passes the tests.
History
Date User Action Args
2016-06-06 15:09:51A. Skrobovsetrecipients: + A. Skrobov, fdrake, brett.cannon, rhettinger, paul.moore, vstinner, giampaolo.rodola, christian.heimes, tim.golden, benjamin.peterson, berker.peksag, zach.ware, serhiy.storchaka, eryksun, steve.dower
2016-06-06 15:09:50A. Skrobovsetmessageid: <1465225790.32.0.354602042327.issue26415@psf.upfronthosting.co.za>
2016-06-06 15:09:50A. Skrobovlinkissue26415 messages
2016-06-06 15:09:50A. Skrobovcreate