This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author samm
Recipients samm
Date 2018-03-20.16:12:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1521562363.32.0.467229070634.issue33110@psf.upfronthosting.co.za>
In-reply-to
Content
Whilst working with concurrent.futures and ThreadPoolExecutors, my colleague and I have noted some undocumented behaviour.

When adding a done_callback to a future that has already completed, we note that that callback is executed directly, outside of any try...except statement.  However, both the docs and the _invoke_callbacks methods will wrap any done_callbacks in a try...except statement which logs the result and returns.

Could we either update the documentation to mention that callback behaviour may raise if added to an already completed future, or simply add the same try...except wrapping that the _invoke_callback method already uses please? (Preferably the latter)

The two pieces of the futures library I am referring to can be viewed here:
_invoke_callbacks: https://github.com/python/cpython/blob/master/Lib/concurrent/futures/_base.py#L323

add_done_callback: https://github.com/python/cpython/blob/master/Lib/concurrent/futures/_base.py#L403

I would note, that the test code which covers this area of the code, doesn't currently exercise this particular condition.  The closest test I could find is test_done_callback_already_failed, which checks that a callback can retrieve an exception from a future, but it does not validate what happens when a callback raises when the future it is attached to is already complete.
Source: https://github.com/python/cpython/blob/c3d9508ff22ece9a96892b628dd5813e2fb0cd80/Lib/test/test_concurrent_futures.py#L1012

The other test closely related is test_done_callback_raises, however this doesn't check the behaviour of a callback when added to an already completed future.  We should be able to simulate this by moving the f.set_result line to above the f.add_done_callback lines?

Source: https://github.com/python/cpython/blob/c3d9508ff22ece9a96892b628dd5813e2fb0cd80/Lib/test/test_concurrent_futures.py#L990
History
Date User Action Args
2018-03-20 16:12:43sammsetrecipients: + samm
2018-03-20 16:12:43sammsetmessageid: <1521562363.32.0.467229070634.issue33110@psf.upfronthosting.co.za>
2018-03-20 16:12:43sammlinkissue33110 messages
2018-03-20 16:12:43sammcreate