classification
Title: urllib.request and http.client should allow certificate checking
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: 1589 Superseder:
Assigned To: orsenthil Nosy List: Ryan.Tucker, ahasenack, asdfasdfasdfasdfasdfasdfasdf, debatem1, devin, giampaolo.rodola, heikki, janssen, jsamuel, kiilerix, orsenthil, pitrou, python-dev, vila, zooko
Priority: normal Keywords: patch

Created on 2010-06-16 01:04 by debatem1, last changed 2014-11-23 17:46 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
httpcli.patch pitrou, 2010-10-08 16:56
httpcli2.patch pitrou, 2010-10-09 15:09
httpcli+urllib.patch pitrou, 2010-10-11 16:34
httpcli+urllib2.patch pitrou, 2010-10-11 17:47
httpcli+urllib3.patch pitrou, 2010-10-11 19:48
Messages (13)
msg107900 - (view) Author: geremy condra (debatem1) Date: 2010-06-16 01:04
urllib currently blindly accepts bad certificates when passed an https address. This behavior, clearly not desirable for many users, is also not documented. I propose one of two changes:

1) add mechanisms for enforcing correct behavior to urllib, or
2) change the documentation for that module to include something akin to the following warning:

"Warning: urllib does not perform certificate checks if passed an HTTPS url! This permits remote machines to masquerade as your intended destination."
msg118081 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-06 20:52
A big warning is now present (*) in the urllib and httplib documentation pages. Also, once issue1589 is fixed, we can go forward and make {http.client,urllib.request} check hostname and cert if the user gives the location of a bunch of CA certs.

(*) see e.g. http://docs.python.org/dev/library/urllib.request.html
msg118178 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-08 11:03
Here is the API addition I would suggest for the http.client module:

Add two new keyword arguments `context` and `check_hostname` to HTTPSConnection; `context` would allow to pass a SSLContext instance for certificate checking and other options (default None, meaning no checking); `check_hostname` would specify whether to check the hostname against the URL (default to check only if context is present and context.verify_mode != CERT_NONE).

Here is the API addition I would suggest for the urllib.request module:

- Add constructor arguments `context` and `check_hostname` to HTTPSHandler. They will be passed to the underlying HTTPSConnection.

- Add `ssl_ca_file` and `ssl_ca_path` arguments to the high-level function urlopen(); if at least one of them is present, a custom opener with a custom HTTPSHandler will be created, mandating the checking of server certificates
msg118206 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-08 16:56
Here is a preliminary patch for http.client. I think it would be good to have local tests using a custom HTTPS server, too.
msg118276 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-09 15:09
Here is another patch for http.client containing more tests, including with a mismatching cert. Comments welcome.
msg118291 - (view) Author: geremy condra (debatem1) Date: 2010-10-09 21:34
Any chance on folding the HTTPSServer class into http.server?

Geremy Condra
msg118292 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-09 21:56
> Any chance on folding the HTTPSServer class into http.server?

Its API and implementation would first have to be cleaned up.
I'd prefer if it were the subject of a separate issue.
msg118379 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-11 16:34
Here is a patch which also adds 'cafile' and 'capath' keyword arguments to urlopen().
msg118381 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-11 17:47
Here is a new patch with doc updates for urllib.request.
msg118383 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-11 19:48
This patch should fix the test hanging issues witnessed on some machines.
msg118391 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-10-11 20:35
Yes, it does solve the problem of httplib and urllib2_localnet tests which
were hanging with the earlier patch on certain machines..
msg118510 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-13 10:40
Patch committed in r85408. I believe this fixes, at last, the whole issue people were complaining about.
msg231573 - (view) Author: Roundup Robot (python-dev) Date: 2014-11-23 17:46
New changeset 1882157b298a by Benjamin Peterson in branch '2.7':
allow passing cert/ssl information to urllib2.urlopen and httplib.HTTPSConnection
https://hg.python.org/cpython/rev/1882157b298a
History
Date User Action Args
2014-11-23 17:46:30python-devsetnosy: + python-dev
messages: + msg231573
2010-11-11 15:55:14pitroulinkissue8804 superseder
2010-10-13 10:40:17pitrousetstatus: open -> closed
versions: - Python 3.3
messages: + msg118510

resolution: fixed
stage: patch review -> resolved
2010-10-12 18:06:41pitrousetfiles: - unnamed
2010-10-11 20:35:39orsenthilsetfiles: + unnamed

messages: + msg118391
2010-10-11 19:48:53pitrousetfiles: + httpcli+urllib3.patch

messages: + msg118383
2010-10-11 17:47:19pitrousetfiles: + httpcli+urllib2.patch

messages: + msg118381
2010-10-11 16:34:21pitrousetfiles: + httpcli+urllib.patch

messages: + msg118379
stage: needs patch -> patch review
2010-10-09 21:56:55pitrousetmessages: + msg118292
2010-10-09 21:34:16debatem1setmessages: + msg118291
2010-10-09 15:09:40pitrousetfiles: + httpcli2.patch

messages: + msg118276
2010-10-08 16:56:48pitrousetfiles: + httpcli.patch
keywords: + patch
messages: + msg118206
2010-10-08 11:03:36pitrousetresolution: accepted -> (no value)
messages: + msg118178
2010-10-06 20:52:27pitrousettype: enhancement
dependencies: + New SSL module doesn't seem to verify hostname against commonName in certificate

title: urllib about https behavior -> urllib.request and http.client should allow certificate checking
nosy: + zooko, janssen, vila, heikki, ahasenack, kiilerix, devin, asdfasdfasdfasdfasdfasdfasdf, Ryan.Tucker
versions: + Python 3.2, Python 3.3, - Python 3.1
messages: + msg118081
stage: needs patch
2010-06-16 16:54:21jsamuelsetnosy: + jsamuel
2010-06-16 14:24:26giampaolo.rodolasetnosy: + giampaolo.rodola
2010-06-16 03:08:31r.david.murraysetnosy: + pitrou
2010-06-16 01:20:55orsenthilsetassignee: orsenthil

resolution: accepted
nosy: + orsenthil
2010-06-16 01:04:36debatem1create