classification
Title: asyncore creates selec (or poll) on every iteration
Type: performance Stage:
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, haypo, mmarkk
Priority: normal Keywords:

Created on 2011-02-21 15:49 by mmarkk, last changed 2014-06-27 20:39 by haypo. This issue is now closed.

Messages (3)
msg128965 - (view) Author: Марк Коренберг (mmarkk) Date: 2011-02-21 15:49
select.poll() object or r,w,e arrays for select() should not be built from the start in each iteration.

For performance issue, such objects should be created in loop() function and modified in add_channel/del_channel. As we do not know, what type of event loop (select or poll) will be choosed, we must either build both (bad performance anyway) or create new class:

1. new class a-la "poller" should be created, accepting "map" and "use_poll" parameters. poll() should be moved to poller.poll()
2. The "dispatcher" class should accept "poller" as parameter (not just "map")
3. "add_channel" and "del_channel" should add/remove items in "poller" (select.poll.modify or list.remove for r,w,e)
4. "poller" should have weakref to each controlled "dispatcher" just for unregistering dead dispatchers via weakref callback.

I can create a patch, but will not start until someone approve my idea.
msg156439 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-03-20 17:24
I agree it would be great to do this, in fact I'm using a modified version of asyncore supporting register(), unregister() and modify() methods for file descriptors, and the performance benefits are enormous.

On the other hand, it appears to be quite difficult to integrate such a massive change into asyncore in a fully backward compatible manner. At least, it's not clear to me how to do this without breaking code relying on map's parameter and asyncore.socket_map.
msg221722 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-06-27 20:39
"On the other hand, it appears to be quite difficult to integrate such a massive change into asyncore in a fully backward compatible manner. At least, it's not clear to me how to do this without breaking code relying on map's parameter and asyncore.socket_map."

Python 3.4 has now asyncio which creates a selector object which has register/unregister methods and so benefit of performances enhancements of epoll/kqueue/devpoll.

Since Giampaolo cares of backward compatibility of asyncore, and the fact that asyncore is now marked as deprecated ("This module exists for backwards compatibility only. For new code we recommend using asyncio."), I close this issue as wont fix ("wont fix" in asyncore, but it's already fixed in asyncio ;-)).
History
Date User Action Args
2014-06-27 20:39:11hayposetstatus: open -> closed

nosy: + haypo
messages: + msg221722

resolution: wont fix
2012-03-31 22:47:07pitrousetversions: - Python 2.6, Python 3.1, Python 2.7, Python 3.2
2012-03-20 17:24:20giampaolo.rodolasetmessages: + msg156439
2011-02-21 16:20:15giampaolo.rodolasetnosy: + giampaolo.rodola
2011-02-21 15:49:14mmarkkcreate