Author gromgull
Recipients gromgull
Date 2014-02-03.14:27:33
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
When reading large files with fileinput, it will work as expected and only process a line at a time when used normally, but if you add an hook_encoded openhook it will read the whole file into memory before returning the first line. 

Verify by running this program on a large text file: 

import fileinput

for l in fileinput.input(openhook=fileinput.hook_encoded('iso-8859-1')):

and check how much memory it uses. Remove the openhook and memory usage goes down to nothing.
Date User Action Args
2014-02-03 14:27:33gromgullsetrecipients: + gromgull
2014-02-03 14:27:33gromgullsetmessageid: <>
2014-02-03 14:27:33gromgulllinkissue20501 messages
2014-02-03 14:27:33gromgullcreate