New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add run_coroutine_threadsafe() to asyncio #69491
Comments
This is a placeholder bug to reference the PR: python/asyncio#273 by Vincent Michel. |
New changeset 25e05b3e1869 by Guido van Rossum in branch '3.4': New changeset e0db10d8c95e by Guido van Rossum in branch '3.5': New changeset 69829a7fccde by Guido van Rossum in branch 'default': |
It's done. But I am hoping you (or someone) will add docs. |
While I was working on the documentation update, I realized that what we called
The docstring would be "Wrap a coroutine, an awaitable or a future in a concurrent.futures.Future.". The documentation would explain that it works like
I attached an implementation of it. Also, note that I added a In any case, I'll keep working on the documentation update. |
I like this idea. |
I'm against that idea. I don't really see a great important future for this method either way: It's just a little bit of glue between the threaded and asyncio worlds, and people will learn how to use it by finding an example. The existing ensure_future() function is mostly meant as an internal helper, with the understanding that libraries written on top of asyncio might also need this functionality. Basically I want people writing asyncio code not to have to worry about the difference between futures and coroutines, because they can both be awaited for; ensure_future() helps preserve that illusion for code that really needs a future (usually to add a callback). But honestly I *don't* want to encourage flipping back and forth between threads and event loops; I see it as a necessary evil. The name we currently have is fine from the POV of someone coding in the threaded world who wants to hand off something to the asyncio world. Why would someone in the threaded world have an asyncio.future that they need to wait for? That sounds like they're mixing up the two worlds -- or they should be writing asyncio code instead of threaded code. OTOH, I would love to see the documentation update! |
I thought you might be interested to know what the name suggests to a relative newcomer to asyncio. When I saw this issue opened, I though "oh, good, an easy way to submit a coroutine from a thread in my test code, now I don't have to write 'loop.call_soon_threadsafe(loop.create_task, my_coroutine(loop=loop))". Which I suspect you are going to tell me is stupid code, but the point is that that's what the name 'call_coroutine_threadsafe' made me think it was going to do. After reading the linked issue I was very confused about what it was actually going to do, as I've never gotten around to looking up what ensure_future does. If I'm understanding correctly now, I think my assumption was pretty much correct, and the only difference is that I get back something I can actually manipulate if I need to, making it even better, and which the documentation should make clear when I get to read it :) |
I agree, but on the other hand supporting awaitables in addition to coroutines would be great. Can we at least add that? |
@rdm: thanks, you nailed it. :-) @yury: but where would you have gotten the awaitable in the first place? It's easy to see how to get a coroutine -- just define it (with either @coroutine or async def) and call it -- and I think that's the only use case that matters here. I'm unclear how you could have gotten another awaitable (e.g. an asyncio.Future instance) in another thread -- except by an utterly confused user. |
Just a few days ago Andrew Svetlov raised a question on github -- how could he refactor So if you have a library with an API exposed through coroutines, it would be great if you have some freedom do refactor it and keep it compatible with asyncio functions such as |
Well, I still worry that this is just going to encourage more people to try and call awaitables from threaded code, and through some circuitous path of misunderstandings (probably involving StackOverflow :-) end up using this function. (Pretty much the worst-case scenario would be calling run_coroutine_threadsafe() targeting the *current* loop, and then blocking the thread waiting for its result. Deadlock!) |
I attached the first version of the documentation for Also, I think we should add a loop.set_task_factory(lambda loop, coro: i_raise_an_exception) will cause the future returned by except Exception as exc:
if future.set_running_or_notify_cancel():
future.set_exception(exc) inside the callback to notify the future. |
The docs look good. (I assume you've generated the HTML and checked that the output looks good, links are clickable etc.) What do you need to add to the concurrency and multithreading section? I agree on the try/except; can you add that to the same diff? (Merging diffs into three different branches is a pain so I want to have to do as little as possible of it.) |
Should I add a note to explain why the loop argument has to be explicitly passed? (there is a note at the beginning of the
This section provides an example to schedule a coroutine from a different thread using
Do you think the exception should be re-raised for the logger?
All right, should I make another PR on the asyncio github repo as well? |
Oh, that's a good idea. It's sort of clear because it starts with "from a
Oh, yes, definitely update that!
Yes.
No, it's easier for me to copy the changes from CPython into the asyncio |
Can we make 'loop' a required argument (no default None value)? |
loop *is* required for this function. It's just that there's an earlier, general comment describing it as optional for all functions in this section. |
I attached a patch that should sum up all the points we discussed. I replaced the |
I think it's fine. I'm going to commit this now. Thanks again! |
New changeset 54c77fdcdb2e by Guido van Rossum in branch '3.4': New changeset 28fcd7f13613 by Guido van Rossum in branch '3.5': New changeset cba27498a2f7 by Guido van Rossum in branch 'default': |
Thanks! |
I think the 'versionchanged' should say "3.4.4, 3.5.1". We've already had one user confused by the fact that it isn't in 3.5.0. |
Good catch! Fixed in https://hg.python.org/cpython/rev/b34c42e46e7b |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: