Message142319
There's been some work done on the GC some time ago to address this type of pattern (I think to reduce from quadratic complexity to amortized linear complexity). I'm adding Antoine and Martin who were involved in this.
> Here is a short python script that demonstrates the issue, the JSON
> file in this case is 1.2GB large:
A couple remarks:
- please indicate the Python version you're using
- you should put your test in a script, one with gc.disable() at the top, to avoid using any cache that might be used internally by cjson
- you should perform "echo 3 > /proc/sys/vm/drop_cache" before each run to make sure you start with a cold page/buffer cache (see how fast your file got read the second time you called read_json_blob()?) |
|
Date |
User |
Action |
Args |
2011-08-18 14:02:29 | neologix | set | recipients:
+ neologix, loewis, amaury.forgeotdarc, pitrou, vstinner, dsvensson, daniel.urban, docs@python |
2011-08-18 14:02:28 | neologix | set | messageid: <1313676148.95.0.290979143574.issue12775@psf.upfronthosting.co.za> |
2011-08-18 14:02:28 | neologix | link | issue12775 messages |
2011-08-18 14:02:27 | neologix | create | |
|