Message210130
When reading large files with fileinput, it will work as expected and only process a line at a time when used normally, but if you add an hook_encoded openhook it will read the whole file into memory before returning the first line.
Verify by running this program on a large text file:
import fileinput
for l in fileinput.input(openhook=fileinput.hook_encoded('iso-8859-1')):
raw_input()
and check how much memory it uses. Remove the openhook and memory usage goes down to nothing. |
|
Date |
User |
Action |
Args |
2014-02-03 14:27:33 | gromgull | set | recipients:
+ gromgull |
2014-02-03 14:27:33 | gromgull | set | messageid: <1391437653.41.0.935040025319.issue20501@psf.upfronthosting.co.za> |
2014-02-03 14:27:33 | gromgull | link | issue20501 messages |
2014-02-03 14:27:33 | gromgull | create | |
|