Author sbt
Recipients asksol, brandon-rhodes, cool-RR, dholth, jnoller, numbernine, rcoyner, santoso.wijaya, sbt, vsekhar
Date 2011-09-13.13:06:42
SpamBayes Score 5.60289e-08
Marked as misclassified No
Message-id <1315919208.86.0.852232046929.issue8713@psf.upfronthosting.co.za>
In-reply-to
Content
Here is a patch which adds the following functions:

  forking_disable()
  forking_enable()
  forking_is_enabled()
  set_semaphore_prefix()
  get_semaphore_prefix()

To create child processes using fork+exec on Unix, call
forking_disable() at the beginning of the program.

I have tested the patch on Linux (by adding forking_disable() to
test_multiprocessing), and it seems to work.  However, the patch does
not modify test_multiprocessing, and I am not sure of the best way to
do so.  (See below.)

There are some issues with named semaphores.  When forking is
disabled, the name of the semaphore must be left unlinked so that
child processes can use sem_open() on the name.  The patch therefore
delays unlinking the name (only when forking is disabled) until the
original SemLock object is garbage collected or the process which
created it exits.

But if a process is killed without exiting cleanly then the name may
be left unlinked.  This happens, for instance, if I run
test_multiprocessing and then keep hitting ^C until all the processes
exit.  On Linux this leaves files with names like

  /dev/shm/sem.mp-fa012c80-4019-2

which represent leaked semaphores.  These won't be destroyed until the
computer reboots or the semaphores are manually removed (by using
sem_unlink() or by unlinking the entry from the file system).

If some form of this patch is accepted, then the problem of leaked
semaphores needs to be addressed, otherwise the buildbots are likely
run out of named semaphores.  But I am not sure how best to do this in
a platform agnostic way.  (Maybe a forked process could collect names
of all semaphores created, via a pipe.  Then it could try to
sem_unlink() all those names when all write-ends of the pipe are
closed.)
History
Date User Action Args
2011-09-13 13:06:50sbtsetrecipients: + sbt, jnoller, rcoyner, asksol, cool-RR, dholth, brandon-rhodes, santoso.wijaya, numbernine, vsekhar
2011-09-13 13:06:48sbtsetmessageid: <1315919208.86.0.852232046929.issue8713@psf.upfronthosting.co.za>
2011-09-13 13:06:48sbtlinkissue8713 messages
2011-09-13 13:06:48sbtcreate