Author tebeka
Recipients tebeka
Date 2013-10-13.01:42:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1381628578.0.0.047697770755.issue19240@psf.upfronthosting.co.za>
In-reply-to
Content
Currently glob.iglob calls os.listdir internally. Which means that if there are many files in the directory - a big list of them is created in memory.

iglob should try to use readdir and be a "true" iterator, not consuming a lot of memory.

See one possible implementation using ctypes at http://stackoverflow.com/questions/4403598/list-files-in-a-folder-as-a-stream-to-begin-process-immediately
History
Date User Action Args
2013-10-13 01:42:58tebekasetrecipients: + tebeka
2013-10-13 01:42:58tebekasetmessageid: <1381628578.0.0.047697770755.issue19240@psf.upfronthosting.co.za>
2013-10-13 01:42:57tebekalinkissue19240 messages
2013-10-13 01:42:57tebekacreate