I do a lot of urllib2 reads of HTTP URLs.  From time to time, I see the
following exception being thrown:

  File "/usr/local/lib/python2.5/socket.py", line 291, in read
    data = self._sock.recv(recv_size)
  File "/usr/local/lib/python2.5/httplib.py", line 509, in read
    return self._read_chunked(amt)
  File "/usr/local/lib/python2.5/httplib.py", line 548, in _read_chunked
    chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: ''

The chunked reading code does not expect an empty string.  I have
attached a patch which checks for ValueError in this case and sets
chunk_left to 0, which will break from the loop.  I am not entirely sure
if this is the correct fix, however, but it should illustrate the problem.

thanks for the patch.  can you check the HTTP/1.1 RFC and see what it
says  (if anything) about handling this case?

also, do you happen to get this consistently from any particular urls or
servers or is this pretty random just happening under load as you fetch
a bunch of stuff?

i'm wondering if this happens because we just haven't received/read
enough data yet.  (i haven't looked closely at the code yet, hopefully
that is clear by reading it)

Unfortunately I do not get this consistently from any URL.  My code is
periodically polling a single HTTP server over the Internet.  Quite
possibly it is triggered by some random network/remote server condition. 

I have not had the chance to look deeply into the RFC yet.

The trunk version does already handle this: it closes the connection and
raises IncompleteRead.

could you backport that to release25-maint if it isn't there already?

I decided not to do that since it changes the exception raised, which I
guess the release managers won't view as a bugfix.