When reading with a numeric terminator, asynchat assumes that if the server didn't send fewer bytes than requested, that it sent exactly the number of bytes requested. This is normally the case, but an overloaded server can return an error message that is  larger than the number of bytes requested. When this  happens asynchttp.handle_read() goes into a tight loop, returning an empty string for data and zero  for a terminator.

This patch raises an exception if the number of bytes returned exceeds the number requested. It's kind of an ugly fix, because asynchttp doesn't raise any other exceptions, and passes on those it catches to handle_error. However, this really is an exceptional case, and the results of not handling it are disastrous.  

Logged In: YES 
user_id=6399

*** asynchat.pyTue Mar 20 11:11:58 2001
--- asynchat.py.originalTue Mar 20 10:43:01 2001
***************
*** 49,60 ****
  import socket
  import asyncore
 
- class async_chat_exception(Exception):
-     def __init__(self, message):
-         self._message = message
-     def __str__(self):
-         return self._message
-    
  class async_chat (asyncore.dispatcher):
      """This is an abstract class.  You must derive from this class, and add
      the two methods collect_incoming_data() and found_terminator()"""
--- 49,54 ----
***************
*** 111,128 ****
                      self.collect_incoming_data (self.ac_in_buffer)
                      self.ac_in_buffer = ''
                      self.terminator = self.terminator - lb
!                 elif lb == n:
                      self.collect_incoming_data (self.ac_in_buffer[:n])
!                     self.ac_in_buffer = ''
                      self.terminator = 0
                      self.found_terminator()
-                 else:
-                     # if we got back more than we asked for, the server
-                     # is badly confused
-                     raise async_chat_exception(
-                         "Unexpect response: requested %s bytes got %s. %s" % (
-                         n, lb, self.ac_in_buffer
-                         ))
              else:
                  # 3 cases:
                  # 1) end of buffer matches terminator exactly:
--- 105,115 ----
                      self.collect_incoming_data (self.ac_in_buffer)
                      self.ac_in_buffer = ''
                      self.terminator = self.terminator - lb
!                 else:
                      self.collect_incoming_data (self.ac_in_buffer[:n])
!                     self.ac_in_buffer = self.ac_in_buffer[n:]
                      self.terminator = 0
                      self.found_terminator()
              else:
                  # 3 cases:
                  # 1) end of buffer matches terminator exactly:

Logged In: YES 
user_id=6399

I have discovered that there are real cases where the server
sends back more bytes than requested: specifically in
Trannsfer-mode: chunked.  Here is a revised, much simpler
patch that passes on the data.
 *** asynchat.py.original        Tue Mar 20 10:43:01 2001
--- asynchat.py Sun Mar 25 09:13:56 2001
***************
*** 106,113 ****
                      self.ac_in_buffer = ''
                      self.terminator = self.terminator - lb
                  else:
!                     self.collect_incoming_data
(self.ac_in_buffer[:n])
!                     self.ac_in_buffer =
self.ac_in_buffer[n:]
                      self.terminator = 0
                      self.found_terminator()
              else:
--- 106,113 ----
                      self.ac_in_buffer = ''
                      self.terminator = self.terminator - lb
                  else:
!                     self.collect_incoming_data
(self.ac_in_buffer)
!                     self.ac_in_buffer = ''
                      self.terminator = 0
                      self.found_terminator()
              else:

Logged In: YES 
user_id=6380

Doug, I'm hesitant to apply this right before releasing 2.1
-- I have no way to test this code.

Can you ask for a review by Sam Rushing, asynchat.py's
author?

Logged In: YES 
user_id=6399

I have sent a copy of the patch to Sam Rushing with a
request for review.

Logged In: YES 
user_id=6399

Sam Rushing thinks that this patch will break other
protocols.  I have to admit, I was only concerned with
HTTP.  I can work around them problem by overloading
handle_read(), so why don't we forget the whole thing?