This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author sbt
Recipients Olivier.Grisel, lars, sbt
Date 2013-09-10.16:04:39
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1378829080.62.0.248798712771.issue18999@psf.upfronthosting.co.za>
In-reply-to
Content
With your patch, I think if you call get_start_method() without later calling set_start_method() then the helper process(es) will never be started.

With the current code, popen.Popen() automatically starts the helper processes if they have not already been started.

Also, set_start_method() can have the side-effect of starting helper process(es).  I do not really approve of new processes being started as a side-effect of importing a library.  But it is reasonable for a library to want a specific start method unless the user demands otherwise.

I will have to think this over.


BTW, the reason for discouraging using set_start_method() more than once is because some shared resources are created differently depending on what the current start method is.

For instance using the fork method semaphores are created and then immediately unlinked.  But with the other start methods we must not unlink the semaphore until we are finished with it (while being paranoid about cleanup).

Maybe it would be better to have separate contexts for each start method.  That way joblib could use the forkserver context without interfering with the rest of the user's program.

    from multiprocessing import forkserver_context as mp
    l = mp.Lock()
    p = mp.Process(...)
    with mp.Pool() as pool:
        ...
History
Date User Action Args
2013-09-10 16:04:40sbtsetrecipients: + sbt, lars, Olivier.Grisel
2013-09-10 16:04:40sbtsetmessageid: <1378829080.62.0.248798712771.issue18999@psf.upfronthosting.co.za>
2013-09-10 16:04:40sbtlinkissue18999 messages
2013-09-10 16:04:40sbtcreate