It's great that Christian did all the work he did on the SSL module to enhance its security capabilities, and great that Antoine did the work he did before that.  Now we need an explanation of how best to use it all :)


It is not clear from the existing documentation how to best use the various standard library modules that support SSL in a "best practices" way.  Perhaps this could go in the SSL docs and be linked from all the library components that use it.  Alternatively we could perhaps have a general security overview chapter in the library reference, but we at least an SSL one.  The existing documentation in the SSL module, while it contains a lot of information about the available, doesn't make it clear what a programmer should actually *do*.  As one example, it is not clear when or even if an application programmer would call check_hostname.

+1 I think this is sorely needed.

I'm not so sure about the objection. The text currently says

CERT_NONE is the default. Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of time you would like to ensure the authenticity of the server you’re talking to. Therefore, when in client mode, it is highly recommended to use CERT_REQUIRED. However, it is in itself not sufficient; you also have to check that the server certificate, which can be obtained by calling SSLSocket.getpeercert(), matches the desired service. For many protocols and applications, the service can be identified by the hostname; in this case, the match_hostname() function can be used. This common check is automatically performed when SSLContext.check_hostname is enabled.

So from that, you learn that
- check_hostname enables a common check (so you probably should enable it for that reason alone),
- the check involves verifying that the service you requested is the one identified by the hostname

If you are using the SSL module, you should be familiar with the terminology "hostname", "service","authenticate", and "peer", you should also know whether your specific protocol relies on hostnames to identify services. IMO, we *shouldn't* give an introduction to TLS in the Python documentation; that would be doomed as either being incomplete, or over the heads of most readers.

There might be other stuff that actually isn't explained at all, but check_hostname is (IMO) a bad example for that.

Someone else told me it was, I'm afraid I didn't actually check myself :(.

That said, it is still not clear to me from reading that whether or not I need to "do something" when using, say, the SMTP_SSL class.  And it is the latter kind of question that I'm asking be answered, for all of the stdlib modules that support SSL.

To expand on that point a little: in the past, I could happily use the SMTP_SSL class (say) without thinking about certificates or server hostname verification, or pretty much of anything.  This produced no verification, of course, which is the problem we are trying to solve.  So we should have recipes *somewhere* in the docs that show how to use these facilities securely.  It isn't obvious what the default security level currently is.

It is true that, while the ssl docs have (I think) an appropriate discussion of "security considerations", higher-level APIs (i.e. applicative protocols) don't provide any recommendations.

I don't know where we should put them. The recommendations will be similar for most protocols, so it sounds wrong to paste them in every module doc. Perhaps there should be a FAQ entry or a HOWTO?

http://docs.python.org/3.4/library/ssl.html#ssl-security doesn't mention http://docs.python.org/3.4/library/ssl.html#ssl.create_default_context and http://docs.python.org/3.4/library/ssl.html#ssl.SSLContext.check_hostname . I planed to write a paragraph about context but my personal life got into my way (new job, relocation, new apartment).

Can somebody please write a few sentences that explain that:

* no stdlib module verifies SSL cert chain and hostname (except for asyncio)
* developers must pass a correctly configured context to stdlib modules to get validation and hostname matching
* ssl.create_default_context() returns a context with sensible default settings *and* pre-loaded root CA certs on most systems.

Example:

>>> import ssl, smtplib
>>> smtp = smtplib.SMTP("mail.python.org", port=587)
>>> context = ssl.create_default_context()
>>> smtp.starttls(context=context)
(220, b'2.0.0 Ready to start TLS')

Example with missing root CA:

>>> smtp = smtplib.SMTP("mail.python.org", port=587)
>>> context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
>>> context.verify_mode = ssl.CERT_REQUIRED
>>> smtp.starttls(context=context)
Traceback (most recent call last):
...
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)

Here is a patch adding the requested information to the ssl docs.
It doesn't touch the pages for higher-level modules, I'll let someone else decide how to do that.

I'd add a sentence to this paragraph

"""If a client certificate is needed for the connection, it can be added with

context.load_cert_chain(path_to_certificate, path_to_private_key)
"""

Then, in all places where a context and cert/key parameters are allowed (e.g. SMTP_SSL and starttls()), I'd remove the sentence saying that context is an alternative way to provide a private key, and say instead

"""See <link to "best defaults"> for configuring parameters of the TLS connection, such as validation of the server certificate, using the context parameter."""

Updated patch with Martin's suggested changes, and various other nits.

New changeset 8661ffca4581 by Antoine Pitrou in branch '3.4':
Issue #20913: improve the SSL security considerations to first advocate using create_default_context().
http://hg.python.org/cpython/rev/8661ffca4581

New changeset e703fc5262b7 by Antoine Pitrou in branch 'default':
Issue #20913: improve the SSL security considerations to first advocate using create_default_context().
http://hg.python.org/cpython/rev/e703fc5262b7

I've committed the latest patch. I hope things are better now!

I made some review comments.  There is one bug with your patch (you dropped some argument descriptions in one place.)

Ah, I see.  Obviously I didn't read it as carefully as I thought I had ;(.

It's really too bad cert validation fails on that ftp site.  It would be nice to show best practices in that example.  We really need that python test server Benjamin was talking about.

New changeset c38ce7726737 by Antoine Pitrou in branch '3.4':
Issue #20913: make it clear that create_default_context() also enables hostname checking
http://hg.python.org/cpython/rev/c38ce7726737

New changeset 015c4d785be7 by Antoine Pitrou in branch 'default':
Issue #20913: make it clear that create_default_context() also enables hostname checking
http://hg.python.org/cpython/rev/015c4d785be7