This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author dchevell
Recipients dchevell
Date 2019-01-11.08:36:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1547195781.25.0.0195934881381.issue35715@roundup.psfhosted.org>
In-reply-to
Content
ProcessPoolExecutor workers will hold onto the return value of their last task in memory until the next task is received. Since the return value has already been propagated to the parent process's `Future` or else effectively discarded, this is holding onto objects unnecessarily.

Simple case to reproduce:

    import concurrent.futures
    import time

    executor = concurrent.futures.ProcessPoolExecutor(max_workers=1)

    def big_val():
        return [{1:1} for i in range(1, 1000000)]

    executor.submit(big_val)

    # Observe the memory usage of the process worker during the sleep interval
    time.sleep(10)


This should be easily fixed by having the worker explicitly `del r` after calling `_sendback_result` as it already does this for `call_item`
History
Date User Action Args
2019-01-11 08:36:25dchevellsetrecipients: + dchevell
2019-01-11 08:36:21dchevellsetmessageid: <1547195781.25.0.0195934881381.issue35715@roundup.psfhosted.org>
2019-01-11 08:36:21dchevelllinkissue35715 messages
2019-01-11 08:36:21dchevellcreate