classification
Title: urllib2.urlopen() through proxy fails when HTTPS URL contains port number
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Use of set_tunnel with default port results in incorrect post value in host header
View: 22095
Assigned To: serhiy.storchaka Nosy List: atzm, serhiy.storchaka
Priority: normal Keywords:

Created on 2015-05-28 06:04 by atzm, last changed 2015-05-28 19:45 by serhiy.storchaka. This issue is now closed.

Messages (2)
msg244277 - (view) Author: Atzm WATANABE (atzm) Date: 2015-05-28 06:04
urllib2.urlopen() through proxy causes ssl.CertificateError when HTTPS URL contains port number.

Sample code:

$ https_proxy='http://proxy.example.com:8080/' python -c 'import urllib2; urllib2.urlopen("https://www.python.org:443/")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/path/to/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/path/to/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/path/to/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/path/to/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/path/to/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/path/to/lib/python2.7/urllib2.py", line 1194, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/path/to/lib/python2.7/httplib.py", line 1053, in request
    self._send_request(method, url, body, headers)
  File "/path/to/lib/python2.7/httplib.py", line 1093, in _send_request
    self.endheaders(body)
  File "/path/to/lib/python2.7/httplib.py", line 1049, in endheaders
    self._send_output(message_body)
  File "/path/to/lib/python2.7/httplib.py", line 893, in _send_output
    self.send(msg)
  File "/path/to/lib/python2.7/httplib.py", line 855, in send
    self.connect()
  File "/path/to/lib/python2.7/httplib.py", line 1274, in connect
    server_hostname=server_hostname)
  File "/path/to/lib/python2.7/ssl.py", line 352, in wrap_socket
    _context=self)
  File "/path/to/lib/python2.7/ssl.py", line 579, in __init__
    self.do_handshake()
  File "/path/to/lib/python2.7/ssl.py", line 816, in do_handshake
    match_hostname(self.getpeercert(), self.server_hostname)
  File "/path/to/lib/python2.7/ssl.py", line 271, in match_hostname
    % (hostname, ', '.join(map(repr, dnsnames))))
ssl.CertificateError: hostname 'www.python.org:443' doesn't match either of 'www.python.org', 'python.org', 'pypi.python.org', 'docs.python.org', 'testpypi.python.org', 'bugs.python.org', 'wiki.python.org', 'hg.python.org', 'mail.python.org', 'packaging.python.org', 'pythonhosted.org', 'www.pythonhosted.org', 'test.pythonhosted.org', 'us.pycon.org', 'id.python.org'

This problem seems to be caused because urllib2.AbstractHTTPHandler.do_open() calls httplib.HTTPSConnection.set_tunnel() without splitting hostname and port number.

To fix this problem, I suggest applying the patch posted in issue https://bugs.python.org/issue22095 .
This problem looks serious because it means we cannot access HTTPS using various port number at restricted environment (e.g. under the firewall).
msg244343 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-05-28 19:45
Thank you for your report Atzm.
History
Date User Action Args
2015-05-28 19:45:54serhiy.storchakasetstatus: open -> closed

type: behavior
assignee: serhiy.storchaka
messages: + msg244343
superseder: Use of set_tunnel with default port results in incorrect post value in host header
resolution: duplicate
stage: resolved
2015-05-28 06:06:40ned.deilysetnosy: + serhiy.storchaka
2015-05-28 06:04:12atzmcreate