This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author njs
Recipients Rokas K. (rku), crusaderky, djarb, jab, jcea, martin.panter, njs, yselivanov, zzzeek
Date 2020-07-07.19:20:10
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1594149611.1.0.756942144967.issue22239@roundup.psfhosted.org>
In-reply-to
Content
Yeah, writing a trivial "event loop" to drive actually-synchronous code is easy. Try it out:

-----

async def f():
    print("hi from f()")
    await g()

async def g():
    print("hi from g()")

# This is our event loop:
coro = f()
try:
    coro.send(None)
except StopIteration:
    pass

-----

I guess there's technically some overhead, but it's tiny.

I think dropping 'await' syntax has two major downsides:

Downside 1: 'await' shows you where context switches can happen: As we know, writing correct thread-safe code is mind-bendingly hard, because data can change underneath your feet at any moment. With async/await, things are much easier to reason about, because any span of code that doesn't contain an 'await' is automatically atomic:

---
async def task1():
    # These two assignments happen atomically, so it's impossible for
    # another task to see 'someobj' in an inconsistent state.
    someobj.a = 1
    someobj.b = 2
---

This applies to all basic operations like __getitem__ and __setitem__, arithmetic, etc. -- in the async/await world, any combination of these is automatically atomic.

With greenlets OTOH, it becomes possible for another task to observe someobj.a == 1 without someobj.b == 2, in case someobj.__setattr__ internally invoked an await_(). Any operation can potentially invoke a context switch. So debugging greenlets code is roughly as hard as debugging full-on multithreaded code, and much harder than debugging async/await code.

This first downside has been widely discussed (e.g. Glyph's "unyielding" blog post), but I think the other downside is more important:

- 'await' shows where cancellation can happen: Synchronous libraries don't have a concept of cancellation. OTOH async libraries *are* expected to handle cancellation cleanly and correctly. This is *not* trivial. With your sqlalchemy+greenlets code, you've introduced probably hundreds of extra unwinding paths that you've never tested or probably even thought about. How confident are you that they all unwind correctly (e.g. without corrupting sqlalchemy's internal state)? How do you plan to even find them, given that you can't see the cancellation points in your code? How can your users tell which operations could raise a cancelled exception?

AFAICT you can't reasonably build systems that handle cancellation correctly without some explicit mechanism to track where the cancellation can happen. There's a ton of prior art here and you see this conclusion over and over.

tl;dr: I think switching from async/await -> greenlets would make it much easier to write programs that are 90% correct, and much harder to write programs that are 100% correct. That might be a good tradeoff in some situations, but it's a lot more complicated than it seems.
History
Date User Action Args
2020-07-07 19:20:11njssetrecipients: + njs, jcea, djarb, zzzeek, jab, martin.panter, yselivanov, Rokas K. (rku), crusaderky
2020-07-07 19:20:11njssetmessageid: <1594149611.1.0.756942144967.issue22239@roundup.psfhosted.org>
2020-07-07 19:20:11njslinkissue22239 messages
2020-07-07 19:20:10njscreate