Author andreasvc
Recipients andreasvc
Date 2014-05-03.23:38:07
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1399160288.58.0.274862722075.issue21423@psf.upfronthosting.co.za>
In-reply-to
Content
It would be useful if concurrent.futures.ThreadPoolExecutor took an initializer argument, like multiprocessing.Pool.

This is useful for example to load a large dataset once upon initialization of each worker process, without have to pass the dataset as an argument with every job submission, which requires serialization.

concurrent.futures has some advantages over multiprocessing such as detecting killed processes ( http://bugs.python.org/issue9205 ), so it would be good if the advantages of both can be combined.

It appears that the following fork of concurrent.futures has added these arguments: https://github.com/enthought/encore/blob/7101984bc384da8e7975876ca2809cc0103c3efc/encore/concurrent/futures/enhanced_thread_pool_executor.py
History
Date User Action Args
2014-05-03 23:38:08andreasvcsetrecipients: + andreasvc
2014-05-03 23:38:08andreasvcsetmessageid: <1399160288.58.0.274862722075.issue21423@psf.upfronthosting.co.za>
2014-05-03 23:38:08andreasvclinkissue21423 messages
2014-05-03 23:38:07andreasvccreate