urllib currently blindly accepts bad certificates when passed an https address. This behavior, clearly not desirable for many users, is also not documented. I propose one of two changes:

1. add mechanisms for enforcing correct behavior to urllib, or
2. change the documentation for that module to include something akin to the following warning:

"Warning: urllib does not perform certificate checks if passed an HTTPS url! This permits remote machines to masquerade as your intended destination."

A big warning is now present (*) in the urllib and httplib documentation pages. Also, once bpo-1589 is fixed, we can go forward and make {http.client,urllib.request} check hostname and cert if the user gives the location of a bunch of CA certs.

(*) see e.g. http://docs.python.org/dev/library/urllib.request.html

Here is the API addition I would suggest for the http.client module:

Add two new keyword arguments context and check_hostname to HTTPSConnection; context would allow to pass a SSLContext instance for certificate checking and other options (default None, meaning no checking); check_hostname would specify whether to check the hostname against the URL (default to check only if context is present and context.verify_mode != CERT_NONE).

Here is the API addition I would suggest for the urllib.request module:

• Add constructor arguments context and check_hostname to HTTPSHandler. They will be passed to the underlying HTTPSConnection.

• Add ssl_ca_file and ssl_ca_path arguments to the high-level function urlopen(); if at least one of them is present, a custom opener with a custom HTTPSHandler will be created, mandating the checking of server certificates

Here is a preliminary patch for http.client. I think it would be good to have local tests using a custom HTTPS server, too.

Here is another patch for http.client containing more tests, including with a mismatching cert. Comments welcome.

Any chance on folding the HTTPSServer class into http.server?

Geremy Condra

Any chance on folding the HTTPSServer class into http.server?

Its API and implementation would first have to be cleaned up.
I'd prefer if it were the subject of a separate issue.

Here is a patch which also adds 'cafile' and 'capath' keyword arguments to urlopen().

Here is a new patch with doc updates for urllib.request.

This patch should fix the test hanging issues witnessed on some machines.

Yes, it does solve the problem of httplib and urllib2_localnet tests which
were hanging with the earlier patch on certain machines..

Patch committed in r85408. I believe this fixes, at last, the whole issue people were complaining about.

New changeset 1882157b298a by Benjamin Peterson in branch '2.7':
allow passing cert/ssl information to urllib2.urlopen and httplib.HTTPSConnection
https://hg.python.org/cpython/rev/1882157b298a