This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients neologix, vstinner
Date 2011-06-26.20:11:06
SpamBayes Score 5.2735594e-15
Marked as misclassified No
Message-id <1309119059.31782.12.camel@marge>
In-reply-to <BANLkTikOq5WAFwyMQuy2jq3X1NDPC12AFg@mail.gmail.com>
Content
> > In which case is Python the leader of the group? ...
> 
> Yes, it's the case by default when you launch a process through a shell.

subprocess doesn't use a shell by default, and I don't think that
multiprocessing uses a shell to start Python.

> > The subprocess maintains a list of the create subprocesses: subprocess.alive, but you need a reference a reference to this list (or you can access it using the Python namespace, but it requires the GIL and you cannot trust the GIL on a crash).
> > Does multiprocessing maintain a list of child processes?
> 
> Yes, we don't have any guarantee about the interpreter's state, and
> furthermore this won't work for processes calling fork() directly.

I don't think that we can have a reliable, generic and portable solution
for this issue. I suggest to only focus on one use case (debug the
multiprocessing and/or subprocess module), and latter try to support
more cases.

I agree that interpreter state can be inconsistent, but faulthandler
does already use read the interpreter state. We cannot do better than
"best effort". Well, it doesn't really matter if faulthandler crashs,
the program is already dying ;-)

To simplify the implementation, I propose to patch multiprocessing
and/or subprocess to register the pid of the child process in a list in
the faulthandler module.

It would be better if these modules unregister pid when a subprocess
exits, but it's not mandatory. We can send a signal to a non existant
process. In the worst case, on a heavy loaded computer, another process
may get the same pid, but it's unlikely. I'm quite sure that
multiprocessing and subprocess already handle the subprocess exit, so it
should be quite simply to add a hook.

> > subprocess can execute any program, not only Python.
> > Send an arbitrary signal to a child process can cause issues.
> Well, faulthandler is disabled by default, no ?

Yes, but I prefer to interfer with unrelated processes if it's possible.

> > By the way, which signal do you want to send to the child processes?
> 
> Hum, SIGTERM maybe? Don't you register some fatal signals by default?

faulthandler.enable() installs a signal handler for SIGSEGV, SIGBUS,
SIGILL and SIGABRT signals. (SIGKILL cannot be handled by the
application.)

> > A test may replace the signal handler of your signal

Well, it's doesn't really matter. If one child process doesn't print the
traceback, you have less information, but it is unlikely and we may get
the information manually or by changing temporary the signal number.
History
Date User Action Args
2011-06-26 20:11:07vstinnersetrecipients: + vstinner, neologix
2011-06-26 20:11:07vstinnerlinkissue12413 messages
2011-06-26 20:11:06vstinnercreate