Message 396809 - Python tracker

➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author	Thomas
Recipients	Thomas, cuibonobo, eric.smith, kevinmehall, malin, serhiy.storchaka
Date	2021-07-01.06:54:55
SpamBayes Score	-1.0
Marked as misclassified	Yes
Message-id	<1625122495.76.0.984588360182.issue42369@roundup.psfhosted.org>
In-reply-to

Content
The monkey patch works for me! Thank you very much! (I have only tested reading, not writing). However, the lock contention of Python's ZipFile is so bad that using multiple threads actually makes the code run _slower_ than single threaded code when reading a zip file with many small files. For this reason, I am not using ZipFile any longer. Instead, I have implemented a subset of the zip spec without locks, which gives me a speedup of over 2500 % for reading many small files compared to ZipFile. I think that the architecture of ZipFile should be reconsidered, but this exceeds the scope of this issue.

The monkey patch works for me! Thank you very much! (I have only tested reading, not writing).

However, the lock contention of Python's ZipFile is so bad that using multiple threads actually makes the code run _slower_ than single threaded code when reading a zip file with many small files. For this reason, I am not using ZipFile any longer. Instead, I have implemented a subset of the zip spec without locks, which gives me a speedup of over 2500 % for reading many small files compared to ZipFile.

I think that the architecture of ZipFile should be reconsidered, but this exceeds the scope of this issue.

History
Date	User	Action	Args
2021-07-01 06:54:55	Thomas	set	recipients: + Thomas, eric.smith, serhiy.storchaka, malin, cuibonobo, kevinmehall
2021-07-01 06:54:55	Thomas	set	messageid: <1625122495.76.0.984588360182.issue42369@roundup.psfhosted.org>
2021-07-01 06:54:55	Thomas	link	issue42369 messages
2021-07-01 06:54:55	Thomas	create