Title: unable to discover preferred HTTPConnection class
Type: enhancement Stage: test needed
Components: Library (Lib) Versions: Python 3.4
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: orsenthil, samwyse
Priority: normal Keywords:

Created on 2013-02-18 21:21 by samwyse, last changed 2015-02-13 17:54 by demian.brecht.

Messages (1)
msg182343 - (view) Author: Samwyse (samwyse) Date: 2013-02-18 21:21
When a URL is opened, the opener-director is responsible for locating the proper handler for the specified protocol. Frequently, an existing protocol handler will be subclassed and then added to the collection maintained by the director. When urlopen is called, the specified request is immediately handed off to the director's "open" method which finds the correct handler and invokes the protocol-specific XXX_open method. At least in the case of the HTTP protocols, if an error occurs then the director is called again to find and invoke a handler for the error; these handlers generally open a new connection after adding headers to avoid the error going forward. Finally, it is important to note that at the present time, the HTTP handlers in urllib2 are built using a class (infourl) that isn't prepared to deal with a persistent connection, so they always add a "Connection: close" header to the request.
Unfortunately, NTLM only certifies the current connection, meaning that a "Connection: keep-alive" header must be used to keep it open throughout the authentication process. Furthermore, because the opener director only provides a do_open method, there is no way to discover the type of connection without also opening it. This means that the HTTPNtlmAuthHandler cannot use the normal HTTPHandler and must therefore must hardcode the HTTPConnection class. If a custom class is required for whatever reason, the only way to cause it to be used is to monkey-patch the code. For an example, see

This can be avoided if, instead of putting the instantiation of the desired HTTP connection class inline, the HTTPHandler classes used a class instance variable. Something like the following should work without breaking any existing code:

class HTTPHandler(AbstractHTTPHandler):

    _connection = httplib.HTTPConnection
    def connection(self):
        """Returns the class of connection being handled."""
        return self._connection

    def http_open(self, req):
        return self.do_open(_connection, req)

    http_request = AbstractHTTPHandler.do_request_
Date User Action Args
2015-02-13 17:54:39demian.brechtsetnosy: - demian.brecht
2014-06-24 05:33:14demian.brechtsetnosy: + demian.brecht
2013-02-22 18:30:19terry.reedysetnosy: + orsenthil
stage: test needed
type: enhancement

versions: - Python 2.6, 3rd party, Python 3.1, Python 2.7, Python 3.2, Python 3.3, Python 3.5
2013-02-18 21:21:11samwysecreate