classification
Title: in HTTPConnection the are len(body) and TypeError catch exception to detect if body is file like object, this hack do work with StringIO object
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Content-length is incorrect when request body is a list or tuple
View: 23350
Assigned To: Nosy List: harobed, martin.panter, orsenthil, r.david.murray
Priority: normal Keywords:

Created on 2011-06-13 17:50 by harobed, last changed 2015-04-13 07:06 by martin.panter. This issue is now closed.

Messages (10)
msg138264 - (view) Author: harobed (harobed) Date: 2011-06-13 17:50
Hi,

in httplib.HTTPConnection._send_request (Python 2.6)
in httplib.HTTPConnection._set_content_length (Python 2.7)
and http.client.HTTPConnection._set_content_length (Python 3.3)

there are something like that :

        try:
            thelen = str(len(body))
        except TypeError as te:
            # If this is a file-like object, try to
            # fstat its file descriptor
            try:
                thelen = str(os.fstat(body.fileno()).st_size)
            except (AttributeError, OSError):
                # Don't send a length if this failed
                if self.debuglevel > 0: print("Cannot stat!!")


If I put StringIO object in body and I do :

>>> len(body)
AttributeError: StringIO instance has no attribute '__len__'

I haven't TypeError.

I think we need to replace :

        try:
            thelen = str(len(body))
        except TypeError as te:

by :

        if hasattr(body, "read"):
           ... # it's file-like object
        else:
           ... # it's string object

What do you think about ?
msg138272 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-13 19:36
fileno and stat won't work on a StringIO object.  So StringIO would need to be special cased to call getvalue.
msg138275 - (view) Author: harobed (harobed) Date: 2011-06-13 19:56
> fileno and stat won't work on a StringIO object.  So StringIO would need to be special cased to call getvalue.

Yes, but it isn't important in chunk transfer encoding.
msg138276 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-13 20:42
But if the len information is available, why not return it?
msg138358 - (view) Author: harobed (harobed) Date: 2011-06-15 07:42
> But if the len information is available, why not return it?

I use HTTPConnection to simulate Apple Finder WebDAV client.
When this WebDAV client do PUT request, it transmit data in chunked encoding mode and not set Content-Length HTTP field.

Do you understand my context ?

Regards,
Stephane
msg138368 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-15 13:41
The code you are suggesting patching is trying its best to return a length.  If that code needs to be fixed to not throw an error when confronted with a StringIO, then it should do its best to return a length.  Your original message on the ticket did not mention chunked encoding, and indeed that appears to be correct.  This bug doesn't appear, from what you have written and what I see in the 3.x code, to have anything to do with chunked encoding.
msg138692 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2011-06-20 08:36
Hello Stephane,

> I use HTTPConnection to simulate Apple Finder WebDAV client.  When
> this WebDAV client do PUT request, it transmit data in chunked
> encoding mode and not set Content-Length HTTP field.
> 
> Do you understand my context ?

Is the server at fault in not returning Content-Length?

Can you provide a full working snippet which demonstrates the bug you
are facing? I fear, if I am lacking some information to completely
determine what needs to be fixed (I understand the code change you
suggested, but also need to know why and under what practical
situation would code flow through that).
msg185930 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2013-04-03 15:10
@harobed if you still have a problem please provide more data as requested in msg138692.
msg240562 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-04-12 16:03
Length computation is being dealt with in issue 23350.
msg240589 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-13 07:06
For the record, it looks like there were a few distinct but related problems here:

* Passing any StringIO object as the body is not supported (see <https://docs.python.org/2/library/httplib.html#httplib.HTTPConnection.request>), because no StringIO class implements the fileno() method.

* Passing a Python 2 natively implemented StringIO.StringIO instance triggers the same AttributeError exception as in Issue 15267, incompatibility with TemporaryFile objects.

* Even if HTTPConnection.request() did support calculating Content-Length for StringIO objects, it would still be wrong for chunk-encoded bodies. I think the lower-level endheaders() or send() methods should be used instead. Or see Issue 12319, about adding chunked encoding for request().
History
Date User Action Args
2015-04-13 07:06:18martin.pantersetnosy: + martin.panter
messages: + msg240589
2015-04-12 16:03:07r.david.murraysetstatus: open -> closed
superseder: Content-length is incorrect when request body is a list or tuple
messages: + msg240562

resolution: duplicate
stage: needs patch -> resolved
2014-02-03 18:35:00BreamoreBoysetnosy: - BreamoreBoy
2013-04-03 15:10:49BreamoreBoysetnosy: + BreamoreBoy
messages: + msg185930
2011-06-20 08:36:52orsenthilsetmessages: + msg138692
2011-06-15 13:41:37r.david.murraysettype: behavior
messages: + msg138368
stage: needs patch
2011-06-15 07:42:56harobedsetmessages: + msg138358
2011-06-13 20:42:47r.david.murraysetversions: + Python 3.2, - Python 2.6
2011-06-13 20:42:20r.david.murraysetmessages: + msg138276
2011-06-13 19:56:52harobedsetmessages: + msg138275
2011-06-13 19:36:36r.david.murraysetnosy: + r.david.murray, orsenthil
messages: + msg138272
2011-06-13 17:50:08harobedcreate