I came across this bug when trying to use CouchDB-Python to PUT an
attachment using HTTP authentication (using the httplib2 library). 
Essentially what httplib2 does is this:

1. Attempt PUT with no auth credentials
2. If a 401 response is received, try again with credentials

However, CouchDB sends a 401 response as soon as it has consumed the
request headers, and before the request body has been fully sent by the
client.

This triggers a bug in both httplib and httplib2, whereby they raise
exceptions due to a broken pipe being encountered when trying to finish
sending the request body.

It seems that Python's httplib checks for broken pipe errors and closes
the connection entirely when they occur when making a request.  This is
unhelpful if a legitimate response was sent and the socket was closed
early by the server.

The offending try/except handler is in httplib.HttpConnection.send and
has a comment saying:

# send the data to the server. if we get a broken pipe, then close
# the socket. we want to reconnect when somebody tries to send again.

This is wrong, as someone might want to read the response before closing
the socket.

For reference, the CouchDB-Python issue is:
http://code.google.com/p/couchdb-python/issues/detail?id=68

This bug may also be related to: http://bugs.python.org/issue3566

Note: in case I didn't make it clear, the fix is to remove the offending
try/except handler and let the broken pipe error propagate.

Wow!  Old issue.  This behavior was present in Greg's original version
of the code.

I think it makes sense to leave the socket open in this case.  (In
general, I think httplib is too aggressive about closing the socket.) 
I'm checking in a version for py3k, and will get around to backporting
it later.

Committed revision 70643.

jhylton> Committed revision 70643.

I backported the fix to 2.7 (r83122). It's required to fix #9032 (XML-RPC bugà in 2.7.

I don't think that it should be backported to 2.6 or 3.1, so I close the issue.

I know this issue is marked as fixed, and won't be backported to 2.6. But the fix is simple enough to self perform on 2.6. Doing that I have discovered an issue that might still be present with the fix, as it was not discussed here, yet, but is still related.

Whenever one does the same thing, but uses an SSL connection (HTTPS), the socket.error is raised from within ssl.py (line 174).

Even though the send() call in httplib does not misbehave now, when using an SSL encrypted connection the ssl module still throws you in front of a bus.

Guy, if the issue you describe in relevant to py3.x or 2.7, can you open a new issue?

I would open a new issue if I had it verified. But I can't verify it right now, as I cannot install the dependencies for a Py2.7 (or 3.x) for running my failing code on the server (running Ubuntu Lucid/LTS).

Unless you're saying I should open an issue for it anyway, without having it verified on 2.7 or 3.x ...

I can report that this occurs on Python 2.7.3, when using urllib[x] wrapped in 'requests'.  Trying to access a site using basic auth over https will not work with certain servers, whereas it will work over plain http.

I tried searching to see if a corresponding https issue exists, but this came up instead.  :-)