Title: asyncore creates selec (or poll) on every iteration
Type: performance Stage:
Components: Library (Lib) Versions: Python 3.3
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, mmarkk
Priority: normal Keywords:

Created on 2011-02-21 15:49 by mmarkk, last changed 2012-03-31 22:47 by pitrou.

Messages (2)
msg128965 - (view) Author: Марк Коренберг (mmarkk) Date: 2011-02-21 15:49
select.poll() object or r,w,e arrays for select() should not be built from the start in each iteration.

For performance issue, such objects should be created in loop() function and modified in add_channel/del_channel. As we do not know, what type of event loop (select or poll) will be choosed, we must either build both (bad performance anyway) or create new class:

1. new class a-la "poller" should be created, accepting "map" and "use_poll" parameters. poll() should be moved to poller.poll()
2. The "dispatcher" class should accept "poller" as parameter (not just "map")
3. "add_channel" and "del_channel" should add/remove items in "poller" (select.poll.modify or list.remove for r,w,e)
4. "poller" should have weakref to each controlled "dispatcher" just for unregistering dead dispatchers via weakref callback.

I can create a patch, but will not start until someone approve my idea.
msg156439 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-03-20 17:24
I agree it would be great to do this, in fact I'm using a modified version of asyncore supporting register(), unregister() and modify() methods for file descriptors, and the performance benefits are enormous.

On the other hand, it appears to be quite difficult to integrate such a massive change into asyncore in a fully backward compatible manner. At least, it's not clear to me how to do this without breaking code relying on map's parameter and asyncore.socket_map.
Date User Action Args
2012-03-31 22:47:07pitrousetversions: - Python 2.6, Python 3.1, Python 2.7, Python 3.2
2012-03-20 17:24:20giampaolo.rodolasetmessages: + msg156439
2011-02-21 16:20:15giampaolo.rodolasetnosy: + giampaolo.rodola
2011-02-21 15:49:14mmarkkcreate