urllib2 sporadically falsely claims that http://www.bankofamerica.com/ has infinite redirect:


$ python -c 'import urllib2; print urllib2.urlopen("http://www.bankofamerica.com/").geturl()'
https://www.bankofamerica.com/

$ python -c 'import urllib2; print urllib2.urlopen("http://www.bankofamerica.com/").geturl()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.6/urllib2.py", line 429, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 605, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.6/urllib2.py", line 429, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 605, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.6/urllib2.py", line 429, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 605, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.6/urllib2.py", line 429, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 605, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.6/urllib2.py", line 429, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 595, in http_error_302
    self.inf_msg + msg, headers, fp)
urllib2.HTTPError: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Found



Since it is sporadic, it could just be a problem with bankofamerica.com's servers. Is there an easy way to see what response urllib2 got that made it unhappy?

Apparently, it's a problem on bankofamerica.com:

$ curl -v https://www.bankofamerica.com/
* About to connect() to www.bankofamerica.com port 443 (#0)
*   Trying 171.161.148.173... connected
* Connected to www.bankofamerica.com (171.161.148.173) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-MD5
* Server certificate:
* 	 subject: 1.3.6.1.4.1.311.60.2.1.3=US; 1.3.6.1.4.1.311.60.2.1.2=Delaware; businessCategory=V1.0, Clause 5.(b); serialNumber=2927442; C=US; postalCode=75202; ST=Texas; L=Dallas; street=1201 Main Street; O=Bank of America Corporation; OU=WebSphere Ecomm; CN=www.banko
* 	 start date: 2010-02-24 00:00:00 GMT
* 	 expire date: 2011-03-06 23:59:59 GMT
* 	 common name: www.bankofamerica.com (matched)
* 	 issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of use at https://www.verisign.com/rpa (c)06; CN=VeriSign Class 3 Extended Validation SSL CA
* 	 SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.20.1 (x86_64-mandriva-linux-gnu) libcurl/7.20.1 OpenSSL/1.0.0a zlib/1.2.3 libidn/1.18 libssh2/1.2.5
> Host: www.bankofamerica.com
> Accept: */*
> 
< HTTP/1.1 302 Found
< Server: Sun-ONE-Web-Server/6.1
< Date: Fri, 07 Jan 2011 19:45:49 GMT
< Content-length: 0
< Content-type: text/html
< Set-Cookie: TLTSID=BAA36B1C1A96101A7D01FB668166E60D; Path=/; Domain=.bankofamerica.com
< Set-Cookie: TLTUID=BAA36B1C1A96101A7D01FB668166E60D; Path=/; Domain=.bankofamerica.com; Expires=Fri, 07-01-2021 19:45:49 GMT
< Location: https://www.bankofamerica.com/
< Content-language: en-US
< Set-cookie: JSESSIONID=0000WgLDVaDtim2XXER98OGo6OA:12rfuebu8; Path=/
< Set-cookie: INTL_LANG=en_US
< Set-cookie: ngen_throttle=750; Expires=Wed, 06 Jul 2011 19:45:49 GMT; Path=/; Domain=.bankofamerica.com
< Set-cookie: hp_beta=B; Expires=Wed, 06 Jul 2011 19:45:49 GMT; Path=/; Domain=.bankofamerica.com
< Set-cookie: BOA_0020=20110107:0:E:961E3F55-9482-01f4-00000000BFA6871D; Expires=Wed, 25 Jan 2079 22:59:56 GMT; Domain=.bankofamerica.com
< Set-cookie: BOA_COM_BT_ELIGIBLE=No; Expires=Fri, 14 Jan 2011 19:45:49 GMT; Domain=.bankofamerica.com
< Expires: Thu, 01 Dec 1994 16:00:00 GMT
< Cache-control: no-cache="set-cookie, set-cookie2"
< 
* Connection #0 to host www.bankofamerica.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):

Actually, perhaps the redirection loop would stop if we would send back those cookies which the server sends us. So perhaps it's a feature request rather.
(but this still strikes me as a very poor use of HTTP)

Ahha, what a mess, thanks for investigating! I agree, this is bankofamerica's problem.

I ran into a redirection loop when cookies were blocked from google, who should know better.  And then they had the gall to blacklist my ip for an attempted DOS attack.  So a feature request may not be out of order; but I for one would not be motivated enough to write a patch for it.