This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: httplib throws ValueError
Type: behavior Stage:
Components: Library (Lib) Versions: Python 2.5
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: georg.brandl Nosy List: georg.brandl, gregory.p.smith, niallo
Priority: normal Keywords: patch

Created on 2008-04-16 18:15 by niallo, last changed 2022-04-11 14:56 by admin. This issue is now closed.

File name Uploaded Description Edit niallo, 2008-04-16 18:15 Diff to fix the issue
Messages (6)
msg65557 - (view) Author: Niall O'Higgins (niallo) Date: 2008-04-16 18:15
I do a lot of urllib2 reads of HTTP URLs.  From time to time, I see the
following exception being thrown:

  File "/usr/local/lib/python2.5/", line 291, in read
    data = self._sock.recv(recv_size)
  File "/usr/local/lib/python2.5/", line 509, in read
    return self._read_chunked(amt)
  File "/usr/local/lib/python2.5/", line 548, in _read_chunked
    chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: ''

The chunked reading code does not expect an empty string.  I have
attached a patch which checks for ValueError in this case and sets
chunk_left to 0, which will break from the loop.  I am not entirely sure
if this is the correct fix, however, but it should illustrate the problem.
msg65559 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2008-04-16 18:21
thanks for the patch.  can you check the HTTP/1.1 RFC and see what it
says  (if anything) about handling this case?

also, do you happen to get this consistently from any particular urls or
servers or is this pretty random just happening under load as you fetch
a bunch of stuff?

i'm wondering if this happens because we just haven't received/read
enough data yet.  (i haven't looked closely at the code yet, hopefully
that is clear by reading it)
msg65627 - (view) Author: Niall O'Higgins (niallo) Date: 2008-04-18 23:32
Unfortunately I do not get this consistently from any URL.  My code is
periodically polling a single HTTP server over the Internet.  Quite
possibly it is triggered by some random network/remote server condition. 

I have not had the chance to look deeply into the RFC yet.
msg65628 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-04-19 08:14
The trunk version does already handle this: it closes the connection and
raises IncompleteRead.
msg65640 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2008-04-20 00:33
could you backport that to release25-maint if it isn't there already?
msg65643 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-04-20 08:00
I decided not to do that since it changes the exception raised, which I
guess the release managers won't view as a bugfix.
Date User Action Args
2022-04-11 14:56:33adminsetgithub: 46897
2008-04-20 08:00:38georg.brandlsetmessages: + msg65643
2008-04-20 00:33:19gregory.p.smithsetmessages: + msg65640
2008-04-19 08:14:57georg.brandlsetstatus: open -> closed
resolution: out of date
messages: + msg65628
2008-04-18 23:32:18niallosetmessages: + msg65627
2008-04-18 22:41:40georg.brandlsetassignee: georg.brandl
nosy: + georg.brandl
2008-04-16 18:21:45gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg65559
2008-04-16 18:15:59niallocreate