This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Michael.Fox
Recipients Michael.Fox, nadeem.vawda
Date 2013-05-17.22:27:23
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
import lzma
count = 0
f = lzma.LZMAFile('bigfile.xz' ,'r')
for line in f:
    count += 1

Comparing python2 with pyliblzma to python3.3.1 with the new lzma:

m@air:~/q/topaz/parse_datalog$ time python

real    0m0.062s
user    0m0.056s
sys     0m0.004s
m@air:~/q/topaz/parse_datalog$ time python3

real    0m7.506s
user    0m7.484s
sys     0m0.012s

Profiling shows most of the time is spent here:

   102371    6.881    0.000    6.972    0.000

I also notice that reading the entire file into memory with is perfectly fast.

I think it has something to do with lack of buffering.
Date User Action Args
2013-05-17 22:27:24Michael.Foxsetrecipients: + Michael.Fox, nadeem.vawda
2013-05-17 22:27:24Michael.Foxsetmessageid: <>
2013-05-17 22:27:24Michael.Foxlinkissue18003 messages
2013-05-17 22:27:23Michael.Foxcreate