classification
Title: Standard Library documentation needs SSL security best practices doc.
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, christian.heimes, dstufft, ezio.melotti, loewis, pitrou, python-dev, r.david.murray, rhettinger, vstinner
Priority: normal Keywords: patch

Created on 2014-03-13 17:56 by r.david.murray, last changed 2014-03-23 18:02 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
ssl_best_doc.patch pitrou, 2014-03-14 12:23 review
ssl_best2.patch pitrou, 2014-03-21 23:56 review
Messages (16)
msg213425 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-13 17:56
It's great that Christian did all the work he did on the SSL module to enhance its security capabilities, and great that Antoine did the work he did before that.  Now we need an explanation of how best to use it all :)


It is not clear from the existing documentation how to best use the various standard library modules that support SSL in a "best practices" way.  Perhaps this could go in the SSL docs and be linked from all the library components that use it.  Alternatively we could perhaps have a general security overview chapter in the library reference, but we at least an SSL one.  The existing documentation in the SSL module, while it contains a lot of information about the available, doesn't make it clear what a programmer should actually *do*.  As one example, it is not clear when or even if an application programmer would call check_hostname.
msg213498 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-03-14 00:09
+1 I think this is sorely needed.
msg213503 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-03-14 01:18
I'm not so sure about the objection. The text currently says

CERT_NONE is the default. Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of time you would like to ensure the authenticity of the server you’re talking to. Therefore, when in client mode, it is highly recommended to use CERT_REQUIRED. However, it is in itself not sufficient; you also have to check that the server certificate, which can be obtained by calling SSLSocket.getpeercert(), matches the desired service. For many protocols and applications, the service can be identified by the hostname; in this case, the match_hostname() function can be used. This common check is automatically performed when SSLContext.check_hostname is enabled.

So from that, you learn that
- check_hostname enables a common check (so you probably should enable it for that reason alone),
- the check involves verifying that the service you requested is the one identified by the hostname

If you are using the SSL module, you should be familiar with the terminology "hostname", "service","authenticate", and "peer", you should also know whether your specific protocol relies on hostnames to identify services. IMO, we *shouldn't* give an introduction to TLS in the Python documentation; that would be doomed as either being incomplete, or over the heads of most readers.

There might be other stuff that actually isn't explained at all, but check_hostname is (IMO) a bad example for that.
msg213507 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-14 01:39
Someone else told me it was, I'm afraid I didn't actually check myself :(.

That said, it is still not clear to me from reading that whether or not I need to "do something" when using, say, the SMTP_SSL class.  And it is the latter kind of question that I'm asking be answered, for all of the stdlib modules that support SSL.
msg213510 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-14 01:48
To expand on that point a little: in the past, I could happily use the SMTP_SSL class (say) without thinking about certificates or server hostname verification, or pretty much of anything.  This produced no verification, of course, which is the problem we are trying to solve.  So we should have recipes *somewhere* in the docs that show how to use these facilities securely.  It isn't obvious what the default security level currently is.
msg213535 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-03-14 11:50
It is true that, while the ssl docs have (I think) an appropriate discussion of "security considerations", higher-level APIs (i.e. applicative protocols) don't provide any recommendations.

I don't know where we should put them. The recommendations will be similar for most protocols, so it sounds wrong to paste them in every module doc. Perhaps there should be a FAQ entry or a HOWTO?
msg213537 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-03-14 11:53
http://docs.python.org/3.4/library/ssl.html#ssl-security doesn't mention http://docs.python.org/3.4/library/ssl.html#ssl.create_default_context and http://docs.python.org/3.4/library/ssl.html#ssl.SSLContext.check_hostname . I planed to write a paragraph about context but my personal life got into my way (new job, relocation, new apartment).

Can somebody please write a few sentences that explain that:

* no stdlib module verifies SSL cert chain and hostname (except for asyncio)
* developers must pass a correctly configured context to stdlib modules to get validation and hostname matching
* ssl.create_default_context() returns a context with sensible default settings *and* pre-loaded root CA certs on most systems.

Example:

>>> import ssl, smtplib
>>> smtp = smtplib.SMTP("mail.python.org", port=587)
>>> context = ssl.create_default_context()
>>> smtp.starttls(context=context)
(220, b'2.0.0 Ready to start TLS')

Example with missing root CA:

>>> smtp = smtplib.SMTP("mail.python.org", port=587)
>>> context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
>>> context.verify_mode = ssl.CERT_REQUIRED
>>> smtp.starttls(context=context)
Traceback (most recent call last):
...
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)
msg213538 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-03-14 12:23
Here is a patch adding the requested information to the ssl docs.
It doesn't touch the pages for higher-level modules, I'll let someone else decide how to do that.
msg213583 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-03-14 18:43
I'd add a sentence to this paragraph

"""If a client certificate is needed for the connection, it can be added with

context.load_cert_chain(path_to_certificate, path_to_private_key)
"""

Then, in all places where a context and cert/key parameters are allowed (e.g. SMTP_SSL and starttls()), I'd remove the sentence saying that context is an alternative way to provide a private key, and say instead

"""See <link to "best defaults"> for configuring parameters of the TLS connection, such as validation of the server certificate, using the context parameter."""
msg214415 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-03-21 23:56
Updated patch with Martin's suggested changes, and various other nits.
msg214490 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-03-22 17:19
New changeset 8661ffca4581 by Antoine Pitrou in branch '3.4':
Issue #20913: improve the SSL security considerations to first advocate using create_default_context().
http://hg.python.org/cpython/rev/8661ffca4581

New changeset e703fc5262b7 by Antoine Pitrou in branch 'default':
Issue #20913: improve the SSL security considerations to first advocate using create_default_context().
http://hg.python.org/cpython/rev/e703fc5262b7
msg214491 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-03-22 17:20
I've committed the latest patch. I hope things are better now!
msg214577 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-23 13:22
I made some review comments.  There is one bug with your patch (you dropped some argument descriptions in one place.)
msg214585 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-23 14:52
Ah, I see.  Obviously I didn't read it as carefully as I thought I had ;(.
msg214586 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-23 14:59
It's really too bad cert validation fails on that ftp site.  It would be nice to show best practices in that example.  We really need that python test server Benjamin was talking about.
msg214592 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-03-23 15:31
New changeset c38ce7726737 by Antoine Pitrou in branch '3.4':
Issue #20913: make it clear that create_default_context() also enables hostname checking
http://hg.python.org/cpython/rev/c38ce7726737

New changeset 015c4d785be7 by Antoine Pitrou in branch 'default':
Issue #20913: make it clear that create_default_context() also enables hostname checking
http://hg.python.org/cpython/rev/015c4d785be7
History
Date User Action Args
2014-03-23 18:02:25r.david.murraysetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2014-03-23 15:31:46python-devsetmessages: + msg214592
2014-03-23 14:59:28r.david.murraysetmessages: + msg214586
2014-03-23 14:52:20r.david.murraysetmessages: + msg214585
2014-03-23 13:22:03r.david.murraysetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg214577

stage: resolved -> needs patch
2014-03-22 17:20:39pitrousetstatus: open -> closed
resolution: fixed
messages: + msg214491

stage: needs patch -> resolved
2014-03-22 17:19:44python-devsetnosy: + python-dev
messages: + msg214490
2014-03-21 23:56:50pitrousetfiles: + ssl_best2.patch

messages: + msg214415
2014-03-21 19:08:24pitrousetnosy: + dstufft
2014-03-20 10:55:20pitroulinkissue20988 superseder
2014-03-14 18:43:40loewissetmessages: + msg213583
2014-03-14 14:48:25Arfreversetnosy: + Arfrever
2014-03-14 12:23:51pitrousetfiles: + ssl_best_doc.patch
keywords: + patch
messages: + msg213538
2014-03-14 11:53:36christian.heimessetmessages: + msg213537
2014-03-14 11:50:04pitrousetmessages: + msg213535
2014-03-14 01:48:01r.david.murraysetmessages: + msg213510
2014-03-14 01:39:36r.david.murraysetmessages: + msg213507
2014-03-14 01:18:07loewissetnosy: + loewis
messages: + msg213503
2014-03-14 00:09:05rhettingersetnosy: + rhettinger
messages: + msg213498
2014-03-13 18:01:55ezio.melottisetnosy: + ezio.melotti
2014-03-13 17:58:21r.david.murraysetassignee: christian.heimes ->
stage: needs patch
type: enhancement
components: + Documentation
versions: + Python 3.4, Python 3.5
2014-03-13 17:56:27r.david.murraycreate