classification
Title: Add metrics to future objects (concurrent or asyncio?)
Type: enhancement Stage:
Components: Library (Lib) Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Joshua.Harlow, bquinlan, giampaolo.rodola, gvanrossum, pitrou, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2015-06-14 17:16 by Joshua.Harlow, last changed 2015-06-15 18:42 by Joshua.Harlow.

Files
File name Uploaded Description Edit
prototype.patch Joshua.Harlow, 2015-06-15 18:42 prototype patch review
Messages (5)
msg245345 - (view) Author: Joshua Harlow (Joshua.Harlow) Date: 2015-06-14 17:16
It would be quite useful to have some types of metrics attached to future objects so that callers could use them for various activities (scheduling repeated runs, post-analysis and such):

Some of the ones that I can think would be useful:

- 'submitted_at' (when the callback was submitted to the executor responsible for executing it)
- 'started_at' (when the executor actually started executing the callback, which is typically sometime after it was submitted)
- 'finished_at' (when the executor finished calling the callback)
- elapsed [finished_at - started_at] (the elapsed runtime of the callback)

Do others feel this would be useful (if so I can put up some patches)? Right now getting this information is pretty hard to get at (since the executors that exist currently do not expose this information and it is quite hard to obtain it in a reliable manner).
msg245370 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2015-06-15 09:32
I'm not sure about concurrent.futures, but for asyncio I think this would
cost too much overhead.
msg245371 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-06-15 09:35
asyncio must be as fast as possible in the default configuration. timestamps look more as a debug feature for me, or a specific use case. I suggested to make asyncio/concurrent.futures "pluggable/hookable" instead of modifying the default configuration. For example, in debug mode, asyncio uses a CoroWrapper class for coroutines which stores more information (ex: traceback where it was created), useless for debug.
msg245383 - (view) Author: Joshua Harlow (Joshua.Harlow) Date: 2015-06-15 15:54
I like the pluggable/hookable idea, that would be nice (I'm siding on the side of hookable, since I think that would be more 'elegant'). If these are just callbacks that can be hooked in for these specific 'events' that would allow me to gather the timing information that is needed (and/or other information as well). Sound like a decent plan to add these?
msg245402 - (view) Author: Joshua Harlow (Joshua.Harlow) Date: 2015-06-15 18:42
A prototype (WIP) of how this could work, initial thoughts welcome :-)
History
Date User Action Args
2015-06-15 18:42:59Joshua.Harlowsetfiles: + prototype.patch
keywords: + patch
messages: + msg245402
2015-06-15 15:54:31Joshua.Harlowsetmessages: + msg245383
2015-06-15 09:35:09vstinnersetmessages: + msg245371
2015-06-15 09:32:12gvanrossumsetmessages: + msg245370
2015-06-15 02:06:12ned.deilysetnosy: + gvanrossum, bquinlan, pitrou, vstinner, giampaolo.rodola, yselivanov
2015-06-14 17:16:11Joshua.Harlowcreate