This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Ericg
Recipients Ericg, ned.deily, ronaldoussoren
Date 2018-02-24.14:17:02
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1519481823.24.0.467229070634.issue32937@psf.upfronthosting.co.za>
In-reply-to
Content
I have the following code:

    import multiprocessing
    from multiprocessing import Pool, Manager
    import time
    import random

    def worker_function( index, messages ):

        print( "%d: Entered" % index )
        time.sleep( random.randint( 3, 15 ) )
        messages.put( "From: %d" % index )
        print( "%d: Exited" % index )

    manager = Manager()
    messages = manager.Queue()

    with Pool( processes = None ) as pool:

        for x in range( 30 ):
            pool.apply_async( worker_function, [ x, messages ] )

        pool.close()
        pool.join()

It does not terminate -- all entered messages are printed, but not all exited messages are printed.

If I remove all the code related to the Manager and Queue, it will terminate properly with all messages printed.

If I assign processes explicitly, I can continue to increase the number assigned to processes and have it continue to work until I reach a value of 20 or 21. > 20, it fails all of the time. With a value == 20 it fails some of the time. With a value of < 20, it always succeeds.

multiprocessing.cpu_count() returns 24 for my MacPro.
History
Date User Action Args
2018-02-24 14:17:03Ericgsetrecipients: + Ericg, ronaldoussoren, ned.deily
2018-02-24 14:17:03Ericgsetmessageid: <1519481823.24.0.467229070634.issue32937@psf.upfronthosting.co.za>
2018-02-24 14:17:03Ericglinkissue32937 messages
2018-02-24 14:17:02Ericgcreate