classification
Title: urrlib2/httplib doesn't reset file position between requests
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ajaksu2, gagenellina, jjlee, matejcik, orsenthil
Priority: normal Keywords: easy

Created on 2009-01-23 17:07 by matejcik, last changed 2009-04-22 17:24 by ajaksu2.

Messages (2)
msg80419 - (view) Author: jan matejek (matejcik) Date: 2009-01-23 17:06
since 2.6 httplib supports reading from file-like objects.

Now consider the following situation:
There are two handlers in urrlib2, first is plain http, second is basic
auth.
I want to POST a file to a service, and pass the open file object as
data parameter to urllib2.urlopen.
First handler is invoked, it sends the file data, but gets 401
Unauthorized return code and fails with that.
Second handler in chain is invoked (at least that's how i understand
urrlib2, please correct me if i'm talking rubbish). At that point the
open file is at EOF, so empty data is sent.

furthermore, the obvious solution "you can't do this through urllib so
go read the file yourself" doesn't apply that well - the file object in
question is actually a mmap.mmap instance.
This code is in production since python 2.4. Until file object support
in httplib was introduced, it worked fine, handling the mmap'ed file as
a string. Now it is picked up as read()-able and this problem occurs.
Only workaround to restore pre-2.6 behavior that comes to mind is
building a wrapper class for the mmap object that hides its read() method.
msg80422 - (view) Author: Gabriel Genellina (gagenellina) Date: 2009-01-23 23:28
This happens in other implementations too, not just urllib2.

If the server supports it, the best way is to send an 'Expect: 100-
Continue' header field before attempting to send the actual file.
History
Date User Action Args
2009-04-22 17:24:08ajaksu2setpriority: normal
keywords: + easy
2009-02-13 01:48:27ajaksu2setnosy: + jjlee
2009-02-12 18:38:39ajaksu2setnosy: + orsenthil, ajaksu2
stage: test needed
2009-01-23 23:28:55gagenellinasetnosy: + gagenellina
messages: + msg80422
2009-01-23 17:07:02matejcikcreate