When using asyncore server with UNIX socket, I got 100% CPU usage.
I run modified code example from asyncore doc page.
This code was tested on two systems:

Ubuntu 10.04 2.6.32-32-generic #62-Ubuntu SMP

with two versions of Python:
Python 3.2 (r32:88445, Mar 29 2011, 08:55:36)

Python 3.2.1rc2 (default, Jul  5 2011, 20:33:19) Built from sources

and
Gentoo 2.6.36-hardened-r9 #6 SMP

with Python 3.1.3 (r313:86834, Mar 12 2011, 20:06:24)

I'm not sure, maybe it's because of the characteristics of UNIX socket?

It's looping in Lib/asyncore.py:poll

select(4, [3], [3], [3], {30, 0})       = 1 (out [3], left {29, 999994})
select(4, [3], [3], [3], {30, 0})       = 1 (out [3], left {29, 999994})
select(4, [3], [3], [3], {30, 0})       = 1 (out [3], left {29, 999994})

loop sets the Unix domain socket in the writable set, and contrarily to AF_INET/AF_INET6 sockets, bound AF_UNIX SOCK_STREAM sockets are reported as writable before any client connects to them, which triggers the loop.

I've attached a patch which just doesn't add the socket to the writable set if it's in the accepting state.
It fixes the loop, and doesn't seem to cause any regression in test_asyncore, but since it's the first time I'm looking at asyncore's code, I might very well have missed something :-)

Looks good, the patch seems to fix the problem.
This section of code indicates that the accepting socket shouldn't be in the write set...

def handle_write_event(self):
    if self.accepting:
        # Accepting sockets shouldn't get a write event.
        # We will pretend it didn't happen.
        return

> Looks good, the patch seems to fix the problem.

Alright.
Giampaolo, do you agree?
Also, I noticed that test_asyncore currently only tests AF_INET sockets.
I'm working a patch to add AF_UNIX (and maybe AF_INET6) tests.

Sorry for delay. Yes, patch looks ok to me and some tests for AF_UNIX sockets would definitively be welcome.

New changeset 16bc59d37866 by Charles-François Natali in branch '2.7':
Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/16bc59d37866

New changeset 42ec507815d2 by Charles-François Natali in branch '3.1':
Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/42ec507815d2

New changeset ed90c1c8ee62 by Charles-François Natali in branch '3.2':
Merge - Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/ed90c1c8ee62

New changeset ca077f2672e3 by Charles-François Natali in branch 'default':
Merge - Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/ca077f2672e3

Woops, I don't know how, but I removed a neologix's message:

"Patch committed, closing.
Alexey, thanks for reporting this.
(I'll open a separate issue to increase the tests coverage)."

Hi,

I am still seeing this issue even with the patches applied. I see 100% CPU utilization, and strace shows the process is in same kind of select() loop as msg139899.

Any help would be greatly appreciated.

Thanks,

-ben

strace output:
"""
select(4, [3], [3], [3], {30, 0})       = 1 (out [3], left {29, 999998})
select(4, [3], [3], [3], {30, 0})       = 1 (out [3], left {29, 999998})
select(4, [3], [3], [3], {30, 0})       = 1 (out [3], left {29, 999998})
select(4, [3], [3], [3], {30, 0})       = 1 (out [3], left {29, 999998})
"""

I'm unable to reproduce it.
Are you using the attached script?
Did you apply the patch by hand, or are you using a recent Python version?

Hi Charles-François,

I am using a recent version of Python-2.7 that does in fact contains this patch http://hg.python.org/cpython/rev/16bc59d37866:
  python-2.7.3-4.fc16.x86_64 (Fedora 16)

The CPU usage spikes after I make the initial client connection to the socket.

I am able to reproduce the issue using the attached script 'asyncore_test.py', and any of the following:
 - simple asyncore.dispatcher_with_send client.
 - simple asyncore.dispatcher client.
 - a direct socket.socket() connection.

Not sure what might be going wrong here. Any help would be most appreciated.

Thanks,

-ben

After doing a bit more testing, I was able to prevent the problem from occurring in asyncore_test.py with the following patch:

--- /proc/self/fd/11	2013-02-04 11:24:41.298347199 -0500
+++ asyncore_test.py	2013-02-04 11:24:40.393318513 -0500
@@ -19,10 +19,18 @@
         self.bind(sock)
         self.listen(5)
 
-    def handle_accepted(self, sock, addr):
-        print('Incoming connection from %s' % repr(addr))
-        handler = EchoHandler(sock)
+    def handle_accept(self):
+        pair = self.accept()
+        if pair is not None:
+            (sock, addr) = pair
+            print('Incoming connection from %s' % repr(addr))
+            handler = EchoHandler(sock)

Using handle_accept() in my code and remembering to call listen() in my asyncore.dispatcher server's constructor did the trick.

I am not sure if we still have a bug here though, since if the subclass doesn't define  a proper handle_accept() we get into the select() loop and 100% CPU utilization after the initial client connection.

> Using handle_accept() in my code and remembering to call listen() in my
> asyncore.dispatcher server's constructor did the trick.
>
> I am not sure if we still have a bug here though, since if the subclass
> doesn't define  a proper handle_accept() we get into the select() loop and
> 100% CPU utilization after the initial client connection.

No, it's not a bug.
The attached test case was for Python 3: Python 2 doesn't have
handle_accepted(), and since the default implementation of
handle_accept() doesn't nothing, the handler is called in a loop,
because the socket is effectively always ready for accept.

Ok, thanks for quick followup. I didn't realize that the patch for Python-3, sorry about that.

The issue  I had was due to never calling self.listen() in the constructor of my server (asyncore.dispatcher). If this is not done the CPU spikes to 100%.

Thanks again.