This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Provide Thread creation hook support
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: christian.heimes, gregory.p.smith, python-dev, stanczyk, twouters
Priority: normal Keywords: patch

Created on 2020-11-23 15:21 by stanczyk, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 23477 open python-dev, 2020-11-23 15:34
Messages (5)
msg381672 - (view) Author: Piotr Stanczyk (stanczyk) * Date: 2020-11-23 15:21
For monitoring purposes we would like to have a way of registering newly created Python threads with a runtime. For that reason we ask to add a thread creation hook inside threading.py. There are 2 similar hooks already (_profile_hook and _trace_hook).
msg381712 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-11-24 08:00
Besindes tests, PR also needs documentation and a better definition how and when the hook is called.

* IMO it should be called after profiling and tracing hook, so non-trivial hooks can be profiled and traced.
* It's important to define and document, which thread runs the hook (calling thread or new thread).
* Since the hook is designed to monitor thread creation, would it make sense to pass the thread object to the hook?
* How does the hook behave when the callback raises an exception?
* Is a single hook good enough or should the API behave more like atexit, which supports an abitrary amount of hooks?
* Instead of just a creation hook, how about lifetime hooks are also called when a thread ends?
msg381755 - (view) Author: Piotr Stanczyk (stanczyk) * Date: 2020-11-24 16:30
Thanks Christian for looking into this, please find my responses inlined:

> * IMO it should be called after profiling and tracing hook, so non-trivial hooks can be profiled and traced.
Makes sense, Done.

> * It's important to define and document, which thread runs the hook (calling thread or new thread).
Will update documentation when we agree upon appropriate API.

> * Since the hook is designed to monitor thread creation, would it make sense to pass the thread object to the hook?
As it's called within the context of the created thread I guess this is not needed (but as you prefer).

> * How does the hook behave when the callback raises an exception?
I guess it makes sense to do similar thing as in case of profile/trace hooks (Error in the profile function will cause itself unset). WDYT?

> * Is a single hook good enough or should the API behave more like atexit, which supports an arbitrary amount of hooks?
Feels like it makes sense to keep API similar to what profile/trace provides, otherwise it would be inconsistent.


> * Instead of just a creation hook, how about lifetime hooks are also called when a thread ends?
Sounds fine, do you mean two independent hooks (_thread_creation_hook, _thread_termination_hook) or some other form?
msg383124 - (view) Author: Piotr Stanczyk (stanczyk) * Date: 2020-12-16 08:42
Friendly ping - does the proposal sound fine?
msg393993 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-05-20 00:38
> > * IMO it should be called after profiling and tracing hook, so non-trivial hooks can be profiled and traced.
> Makes sense, Done.

I updated the PR do this.  (if your "Done" indicated you had done this somewhere, it appears you never pushed your changes to the PR branch on github :)

> > * It's important to define and document, which thread runs the hook 
> (calling thread or new thread).
> Will update documentation when we agree upon appropriate API.

I updated the docstring in the PR.

> > * Since the hook is designed to monitor thread creation, would it make sense to pass the thread object to the hook?
> As it's called within the context of the created thread I guess this is not needed (but as you prefer).

I updated the PR to do this.

> > * How does the hook behave when the callback raises an exception?
> I guess it makes sense to do similar thing as in case of profile/trace hooks (Error in the profile function will cause itself unset). WDYT?

I like the consistency.  I've updated the PR.

> > * Is a single hook good enough or should the API behave more like atexit, which supports an arbitrary amount of hooks?
> Feels like it makes sense to keep API similar to what profile/trace provides, otherwise it would be inconsistent.

Agreed.  when writing the docs we can mention this.  Anyone setting these hooks would be wise to check what's already set and decide if they want to set theirs at all or save that and chain call it or not.  It is expected to be rare to set this.  Our own use case is effectively platform wide for use in most applications.

> > * Instead of just a creation hook, how about lifetime hooks are also called when a thread ends?
> Sounds fine, do you mean two independent hooks (_thread_creation_hook, _thread_termination_hook) or some other form?

Lets leave this as a separate feature to add if the need arises.  I can imagine uses in threading tests, but that doesn't feel like a strong use case.
History
Date User Action Args
2022-04-11 14:59:38adminsetgithub: 86609
2021-05-20 00:38:55gregory.p.smithsetmessages: + msg393993
versions: + Python 3.11
2020-12-16 08:42:09stanczyksetmessages: + msg383124
2020-11-24 16:30:41stanczyksetmessages: + msg381755
2020-11-24 08:00:02christian.heimessetnosy: + christian.heimes
messages: + msg381712
2020-11-23 19:03:37gregory.p.smithsetcomponents: + Library (Lib)
2020-11-23 19:02:43gregory.p.smithsetassignee: gregory.p.smith
2020-11-23 16:16:16twouterssetnosy: + twouters, gregory.p.smith
2020-11-23 15:34:52python-devsetkeywords: + patch
nosy: + python-dev

pull_requests: + pull_request22369
stage: patch review
2020-11-23 15:21:52stanczykcreate