classification
Title: asyncore: included batteries don't fit
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: chrysn, giampaolo.rodola, josiahcarlson, stutzbach, terry.reedy
Priority: normal Keywords:

Created on 2012-09-20 02:59 by chrysn, last changed 2012-09-22 16:47 by chrysn. This issue is now closed.

Messages (5)
msg170782 - (view) Author: chrysn (chrysn) Date: 2012-09-20 02:59
the asyncore module would be much more useful if it were well integrated in the standard library. in particular, it should be supported by:

* subprocess
* BaseHTTPServer / http.server (and thus, socketserver)
* urllib2 / urllib, http.client
* probably many other network libraries except smtpd, which already uses asyncore

without widespread asyncore support, it is not possible to easily integrate different servers and services with each other; with asyncore support, it's just a matter of creating the objects and entering the main loop. (eg, a http server for controlling a serial device, with a telnet-like debugging interface).

the socketserver even documents that it would like to have such a framework ("Future work: [...] Standard framework for select-based multiplexing"). it would be rather difficult to port in the generic case (as socketserver based code in general relies on blocking `read`s or `readline`s), but can be done in particular cases (i've ported SimpleHTTPServer, but it's a mess of monkey-patching). for subprocess, there's a bunch of recipies at [1]; pyserial (not standard library, but might as well become) can be ported quite easily [2].

[1] http://code.activestate.com/recipes/576957-asynchronous-subprocess-using-asyncore/
[2] http://sourceforge.net/tracker/?func=detail&aid=3559321&group_id=46487&atid=446305


this issue touches several modules, so for practical implementation, it might be split up for the different individual fixes to come and track them. some of them will bring incompatible changes -- it's easy to get SimpleHTTPServer to a state where it can run unmodified inside asyncore, and emulate its external interfaces (serve_forever just being a wrapper to asyncore.loop), but hard to say if it will work with what people built atop of it. how can we find a roadmap for that?

python already has batteries for nonblocking operation included -- let's just make sure they fit in the other gadgets!
msg170790 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-09-20 04:52
The proposal looks way too ambitious and vaguely defined to me to say the least.
Integrating any async framework into another one which uses a blocking concurrency model is hard. If that async framework is asyncore is usually also a bad idea.
msg170800 - (view) Author: chrysn (chrysn) Date: 2012-09-20 11:31
i'm aware this is ambitious, and hope that at least the individual sub-agendas will be manageable. as for vague, i can enhance it (i'd start refining the individual sub-agendas -- or do you think the "big picture" needs more details too?).

integration of frameworks is hard, i know. for some libraries, it might not even be feasible, or it could be that it'd be easier to write a new server than integrating into the existing one. (eg it might be easier to refactor http servers into a generic and a blocking part first, and then offer a http.server.AsyncServer in parallel to the existing implementation). that's why i'd like to try to get a rough roadmap instead of hacking ahead :-)

nevertheless, the current situation is not satisfying -- we have a versatile http client module, and yet who wants to fetch asynchronously is left with a stub implementation in the asyncore docs. that's not what one would expect from the "batteries included" catchphrase.

we don't need all of that implemented *right now*, but maybe we can do better than implementing it never.
msg170949 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-09-22 01:28
I think discussion of this idea on python-ideas list would be better. There might be some controversy and alternatives presented. With a positive response, a PEP should probably follow with details on a module by module basis. As this stands, this is not really an actionable issue. So yes, I am suggesting closing for the present.
msg171006 - (view) Author: chrysn (chrysn) Date: 2012-09-22 16:47
i've redirected my request to python-ideas as suggested. for future reference, my email can be found at http://mail.python.org/pipermail/python-ideas/2012-September/016185.html
History
Date User Action Args
2012-09-22 16:47:55chrysnsetmessages: + msg171006
2012-09-22 13:57:29benjamin.petersonsetstatus: open -> closed
resolution: rejected
2012-09-22 01:28:13terry.reedysetnosy: + terry.reedy
messages: + msg170949
2012-09-20 11:31:06chrysnsetmessages: + msg170800
2012-09-20 04:52:20giampaolo.rodolasetmessages: + msg170790
2012-09-20 03:16:38ezio.melottisetnosy: + josiahcarlson, giampaolo.rodola, stutzbach

type: enhancement
components: + Library (Lib)
versions: + Python 3.4
2012-09-20 02:59:43chrysncreate