Author nadeem.vawda
Recipients doko, eric.araujo, eysispeisi, georg.brandl, itkach, koen, lars.gustaebel, loewis, nadeem.vawda, nikratio, paul.moore, pitrou, proyvind, python-dev, v+python, vstinner
Date 2011-12-23.14:51:18
SpamBayes Score 5.73504e-05
Marked as misclassified No
Message-id <1324651879.14.0.0622390311203.issue5689@psf.upfronthosting.co.za>
In-reply-to
Content
Yes, that's a good idea. I've been testing a similar change, and it seems
to drop the peak memory usage for test_tarfile from around 810MB down to
under 200MB. It looks like 2GB genuinely isn't enough to reliably use LZMA
compression with preset=9.

You might want to use preset=None instead of explicitly saying preset=6,
though. This tells LZMAFile to use the default preset, and will allow you
to get rid of the if-statement on lines 1821-1823.

Something unrelated that I noticed in the surrounding code: gzopen and
bz2open validate the mode by testing 'len(mode) > 1 or mode not in "rw"'.
This would be simpler as 'mode not in ("r", "w")' (like you've done in
xzopen), and it would accept only "r" and "w" (but not "" or "rw").
History
Date User Action Args
2011-12-23 14:51:19nadeem.vawdasetrecipients: + nadeem.vawda, loewis, georg.brandl, doko, paul.moore, lars.gustaebel, pitrou, vstinner, koen, eric.araujo, v+python, proyvind, nikratio, itkach, eysispeisi, python-dev
2011-12-23 14:51:19nadeem.vawdasetmessageid: <1324651879.14.0.0622390311203.issue5689@psf.upfronthosting.co.za>
2011-12-23 14:51:18nadeem.vawdalinkissue5689 messages
2011-12-23 14:51:18nadeem.vawdacreate