Issue35792
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2019-01-21 02:59 by chrahunt, last changed 2022-04-11 14:59 by admin.
Messages (4) | |||
---|---|---|---|
msg334109 - (view) | Author: Christopher Hunt (chrahunt) * | Date: 2019-01-21 02:59 | |
Currently AbstractEventLoop.run_in_executor is specified as a coroutine, while BaseEventLoop.run_in_executor is actually a non-coroutine that returns a Future object. The behavior of BaseEventLoop.run_in_executor would be significantly different if changed to align with the interface . If run_in_executor is a coroutine then the provided func will not actually be scheduled until the coroutine is awaited, which conflicts with the statement in PEP 3156 that it "is equivalent to `wrap_future(executor.submit(callback, *args))`". There has already been an attempt in bpo-32327 to convert this function to a coroutine. We should change the interface specified in `AbstractEventLoop` to indicate that `run_in_executor` is not a coroutine, which should help ensure it does not get changed in the future without full consideration of the impacts. |
|||
msg340533 - (view) | Author: Andrew Svetlov (asvetlov) * | Date: 2019-04-19 09:48 | |
I would rather change the implementation by converting it into async function. It can break some code, sure -- but in a very explicit way (coroutine `run_in_executor is never awaited` error). Making existing third-party code forward-compatible is trivial: just push `await` before the call. |
|||
msg340545 - (view) | Author: Christopher Hunt (chrahunt) * | Date: 2019-04-19 14:30 | |
My use case is scheduling work against an executor but waiting on the results later (on demand). If converting `BaseEventLoop.run_in_executor(executor, func, *args)` to a coroutine function, I believe there are two possible approaches (the discussion that started this [here](https://stackoverflow.com/questions/54263558/is-asyncio-run-in-executor-specified-ambiguously) only considers [impl.1]): impl.1) `BaseEventLoop.run_in_executor` still returns a future, but we must await the coroutine object in order to get it (very breaking change), or impl.2) `BaseEventLoop.run_in_executor` awaits on the result of `func` itself and returns the result directly In both cases the provided `func` will only be dispatched to `executor` when the coroutine object is scheduled with the event loop. For [impl.1], from the linked discussion, there is an example of user code required to get the behavior of schedule immediately and return future while still using `BaseEventLoop.run_in_executor`: async def run_now(f, *args): loop = asyncio.get_event_loop() started = asyncio.Event() def wrapped_f(): loop.call_soon_threadsafe(started.set) return f(*args) fut = loop.run_in_executor(None, wrapped_f) await started.wait() return fut however this wrapper would only be possible to use in an async function and assumes the executor is running in the same process - synchronous functions (e.g. an implementation of Protocol.data_received) would need to use an alternative `my_run_in_executor`: def my_run_in_executor(executor, f, *args, loop=asyncio.get_running_loop()): return asyncio.wrap_future(executor.submit(f, *args), loop=loop) either of these would need to be discovered by users and live in their code base. Having to use `my_run_in_executor` would be most unfortunate, given the purpose of `run_in_executor` per the PEP is to be a shorthand for this exact function. For [impl.2], we are fine if the use case allows submitting and awaiting the completion of `func` in the same location, and no methods of asyncio.Future (e.g. `add_done_callback`, `cancel`) are used. If not then we still need to either: soln.1) use `my_run_in_executor`, or soln.2) wrap the `BaseEventLoop.run_in_executor` coroutine object/asyncio.Future with `asyncio.ensure_future` [soln.1] is bad for the reason stated above: this is the function we are trying to avoid users having to write. [soln.2] uses the low-level function `asyncio.ensure_future` because both of the suggested alternatives (per the docs) `asyncio.create_task` and `BaseEventLoop.create_task` throw a `TypeError` when provided an `asyncio.Future` as returned by the current implementation of `BaseEventLoop.run_in_executor`. This will have to be discovered by users and exist in their code base. |
|||
msg340574 - (view) | Author: Christopher Hunt (chrahunt) * | Date: 2019-04-20 14:18 | |
For impl.1: > (very breaking change) should be > (very breaking change, mitigated some by the fact that the implementation will warn about the unawaited future) |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:10 | admin | set | github: 79973 |
2019-04-20 14:18:31 | chrahunt | set | messages: + msg340574 |
2019-04-19 14:30:13 | chrahunt | set | messages: + msg340545 |
2019-04-19 09:48:55 | asvetlov | set | messages: + msg340533 |
2019-01-21 02:59:18 | chrahunt | create |