classification
Title: asyncore kqueue support
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: josiahcarlson Nosy List: Ikinoki, exarkun, giampaolo.rodola, josiahcarlson, lasizoillo, socketpair, vstinner
Priority: normal Keywords: patch

Created on 2009-08-13 01:34 by Ikinoki, last changed 2014-06-27 22:03 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
asyncore.patch giampaolo.rodola, 2010-04-19 20:04
Messages (6)
msg91512 - (view) Author: Andrew Azarov (Ikinoki) Date: 2009-08-13 01:34
Is there a possibility of such feature in the future releases of Python?
Currently I see only select and epoll available, but on FreeBSD 7.2 with
a lot of connections asyncore (1600+ active HTTP connections) has
problems (not giving complete response) with epoll (select is
problematic after 250+ connections (not enough descriptors)).
msg96195 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2009-12-10 08:23
I have a patch for adding epoll() support which is almost ready
(asyncore supports poll(), not epoll()), it's just a matter of writing
tests. 

kqueue() shouldn't be too difficult to implement considering that we can
take Twisted as example.

...But aside from the implementation details I'd say there's a crucial
problem with the current API:

asyncore.loop([timeout[, use_poll[, map[, count]]]])

The "use_poll" argument should be deprecated in favor of something else,
maybe a "poller" which accepts a callable, like this:


>>> asyncore.loop(poller=asyncore.select_poller)
>>> asyncore.loop(poller=asyncore.epoll_poller)
>>> asyncore.loop(poller=asyncore.kqueue_poller)
msg103628 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2010-04-19 19:55
The patch in attachment implements support for epoll() and kqueue() by adding a new "poller" argument to asyncore.loop().

However, I had a chat with Jean Paul Calderone today which pointed out how useless this is. =)
The problem is basically how asyncore.loop() is implemented.
Being impossible to let loop() function know when a file descriptor gets added or removed from the socket map we are forced to iterate over all existing fds on every loop which nullifies the benefits offered by epoll() and kqueue() syscalls.

As of right now I can't think of a solution to this problem which doesn't imply a change to the current asyncore API.
msg103630 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2010-04-19 20:04
Forgot to add the patch.
msg221681 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-27 11:56
Since Python 3.4 has asyncio which supports all selectors provided by the new selectors module (which includes kqueue, but also devpoll since Python 3.5), I propose to close this issue as wontfix since there is no activity since 4 years.
msg221746 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-27 22:03
I read asyncore.patch: it is close to the selectors module, so it means duplicated efforts. I prefer to close this issuse since asyncore has been deprecated in favor of asyncio (and selectors).

Using the selectors module in asyncore would not be efficient because asyncore design requires to build a new selector for each poll, which is not efficient. asyncio only creates the selector once, and then use register/unregister. It's more efficient and scalable.
History
Date User Action Args
2014-06-27 22:03:05vstinnersetstatus: open -> closed
resolution: wont fix
messages: + msg221746
2014-06-27 11:56:49vstinnersetnosy: + vstinner
messages: + msg221681
2011-02-21 17:05:15socketpairsetnosy: + socketpair
2010-04-19 20:04:46giampaolo.rodolasetfiles: + asyncore.patch
keywords: + patch
messages: + msg103630
2010-04-19 19:57:37giampaolo.rodolasetnosy: + exarkun
2010-04-19 19:55:12giampaolo.rodolasetmessages: + msg103628
2010-04-12 21:01:56brian.curtinsetversions: - Python 2.7
2010-04-12 20:59:54lasizoillosetnosy: + lasizoillo
2009-12-10 08:23:52giampaolo.rodolasetmessages: + msg96195
2009-12-10 03:10:19pitrousetpriority: normal
assignee: josiahcarlson

nosy: + josiahcarlson
stage: needs patch
2009-12-09 21:49:42giampaolo.rodolasetnosy: + giampaolo.rodola
2009-08-13 01:34:08Ikinokicreate