This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: ssl module: add getter for SSL_CTX* and SSL*
Type: enhancement Stage: patch review
Components: SSL Versions: Python 3.11, Python 3.10
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, eighthave, njs, steve.dower
Priority: normal Keywords:

Created on 2021-04-21 07:59 by christian.heimes, last changed 2022-04-11 14:59 by admin.

Messages (8)
msg391498 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-21 07:59
Python's ssl module exposes a limited and opinionated set of knobs to tune OpenSSL's behavior. Each new setter, getter, or function must be carefully design, tested, and documented. For each feature OpenSSL's C API must be converted into a Pythonic, self-explanatory interface.

I would like to give experts and power users an interface to set advanced options. libffi-based solutions like ctypes and cffi are obvious choices. For libffi to work, users need to be able to get the address of ssl.SSLContext()'s SSL_CTX pointer and the SSL* pointer of the internal _SSLSocket object.

While it's possible to use pointer arithmetic with id(ctx) + offset, I would like to add a more convenient way. Pointer arithmetic with ctypes is non-trivial. Users would have to rely on internal, private layout of PySSLContext and PySSLSocket struct. I'm considering two new methods ctx._ssl_ctx_addr and ssl._ssl_addr (names are tentative).

>>> import ssl, ctypes
>>> ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
>>> libssl = ctypes.CDLL("")  # ssl._ssl.__file__ works, too
>>> libssl.SSL_CTX_set_ciphersuites(ctx._ssl_ctx_addr(), b"TLS_CHACHA20_POLY1305_SHA256")

Steve, Nathaniel, how do you like the idea in general? Do you have better ideas for function names?
msg391530 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-04-21 17:09
Could we have the address exposed in a way that can only be passed back into ctypes? Or alternatively, doesn't function if ctypes is missing?

I don't like offering ways to get real memory addresses, especially for interesting objects. At the same time, I see the value of it being available raw for CFFI and other tools that can use it.

Maybe it would be better to expose it as ctx._call_with_ctypes("SSL_CTX_set_ciphersuites", b"TLS_CHACHA20_POLY1305_SHA256")? That also leaves the opportunity to special-case certain functions in the future.
msg391533 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2021-04-21 17:36
Funny, I was actually looking at this a bit last week, because I was trying to figure out if I could trick `ssl` into doing DTLS...

The two big problems I ran into are:

- for DTLS you need to instantiate the SSLContext with PROTOCOL_DTLS, and idk how you create a Python ssl._SSLMethod object for a random pointer.

- You need to somehow find the libssl symbols, and it's not clear how to do that in a safe and portable way.

The second issue seems like the big blocker. Your example does `ctypes.CDLL("")`, but there's no guarantee that this will find the same libssl that the `ssl` module is linked against. (And if you get the wrong one then everything will probably explode.)

And like you say, `ctypes.CDLL(ssl._ssl.__file__)` also works, by taking advantage of how ELF shared libraries tend to reexport all the symbols they import, but... this is an ELF-specific thing. It doesn't work on Windows or macOS.

So to me the first question would be: is it possible to offer a utility function that reliably gives users a handle to the right libssl, on all systems and build configurations? If not then I don't see how this could work.
msg391534 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-21 17:41
I don't want to import ctypes from the ssl module code.

PyCapsule could be a solution for the problem. Users would have to call PyCapsule_Import("_ssl.capsule") and PyCapsule_GetPointer() to access a struct with additional methods. It's a bit more work to implement and slightly harder to use, but safer than exposing raw memory addresses.
msg391535 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-21 17:53
Funny, I was looking into the same issue with CDLL(). :)

The trick with ssl._ssl.__file__ may even break if users change sys.setdlopenflag() from RTLD_GLOBAL to RTLD_LOCAL. Static linking will also influence which symbols are available.

Python/dynload_shlib.c keeps a list of dlopen handles around, but there is no way to access the list of handles. Mmh ... tricky.
msg391536 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2021-04-21 17:55
Python's 'id' function exposes raw memory addresses constantly. As long as they're just integers, they can't do much harm.

(In Rust, taking a pointer to a random object is considered totally safe, can be done anywhere. It's *dereferencing* a pointer where you need special 'unsafe' annotations.)

Addresses can potentially reveal ASLR slides or heap layout to an attacker, but I think the marginal risk here is pretty low. You'd need a situation where someone is like, tricking your program into calling ctx._ssl_ctx_addr() and then sending the result to the attacker? Seems unlikely, and not something anyone worries about with 'id'.
msg391539 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-04-21 18:27
There's a working POC out there for modifying tuple elements that relies on the fact that id() returns real memory addresses, so I don't see it as a harmless thing. Also, Rust is only worrying about code that's put into its compiler, whereas Python has to worry about any code that might be injected into a running process, which is vastly easier than in Rust. The situations don't really compare.

It's also part of defense-in-depth. If there's no good reason to make it trivial for people to locate TLS-specific data structures in memory, why make it trivial? It's probably still possible (turns out everything is...), but it's worth raising the complexity and lowering the reliability by making someone jump through crazy hoops to get it.

> The second issue seems like the big blocker. Your example does `ctypes.CDLL("")`, but there's no guarantee that this will find the same libssl that the `ssl` module is linked against.

On Windows, it's spelled "libssl-1_1", but it *is* guaranteed to load the right one. However, I'd rather encode this information somewhere inside _ssl than making it a guarantee (as presumably OpenSSL 3.0 will change this name structure).

Also, this is not the same as ssl._ssl.__file__. So yeah, this aspect is not very portable, which is why I'd rather it just be exposed as a "call OpenSSL function on this context by name".

> I don't want to import ctypes from the ssl module code.

Import it in the function when it's called, then. Reimports are so close to free they're basically irrelevant, and if your tight inner loop includes TLS then it's not actually a tight inner loop, so looking in sys.modules isn't going to hurt anyone.

I would definitely push back on importing it eagerly in :)
msg415606 - (view) Author: Hans-Christoph Steiner (eighthave) Date: 2022-03-20 14:22
This general idea sounds nice to have, I hope it can be included.  `ctx._call_with_ctypes("SSL_CTX_set_ciphersuites"...` also sounds totally workable to me, if that has the best security profile.

Defense in depth is important, but it is not a reason to prevent key functionality from landing.  For example, "export_keying_material" is an RFC and widely implemented (Go crypto/tls, Rustls, Conscrypt, nodejs, boringssl, openssl, BouncyCastle, etc see links here  It is used in IETF protocols like SRTP and NTS.

Perhaps that could be a concrete use case here for thinking about the security profile?
Date User Action Args
2022-04-11 14:59:44adminsetgithub: 88068
2022-03-20 14:22:54eighthavesetnosy: + eighthave
messages: + msg415606
2021-04-21 18:27:37steve.dowersetmessages: + msg391539
2021-04-21 17:55:13njssetmessages: + msg391536
2021-04-21 17:53:23christian.heimessetmessages: + msg391535
2021-04-21 17:41:02christian.heimessetmessages: + msg391534
2021-04-21 17:36:59njssetmessages: + msg391533
2021-04-21 17:09:35steve.dowersetmessages: + msg391530
2021-04-21 07:59:53christian.heimescreate