classification
Title: SSL destructor segfaults in python3.6 threads when an unverified-cert connection is closed
Type: crash Stage: resolved
Components: macOS, SSL Versions: Python 3.6
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, ned.deily, pirate, ronaldoussoren
Priority: normal Keywords:

Created on 2017-01-21 23:37 by pirate, last changed 2017-05-24 00:13 by christian.heimes. This issue is now closed.

Files
File name Uploaded Description Edit
libssl_bug.py pirate, 2017-01-21 23:37 Code to reproduce
Messages (6)
msg285971 - (view) Author: Nick Sweeting (pirate) * Date: 2017-01-21 23:37
I mistakenly thought this bug was a pyOpenSSL bug, so I reported it there (https://github.com/pyca/pyopenssl/issues/588), but it looks like this is actually a bug in CPython.

See the link for a nicer formatted bug report with more details.

Code to reproduce: https://gist.github.com/pirate/6649314e02384274b29e04364c9d0c16

Crash dump:
https://gist.github.com/pirate/e1485110093d2d9fc49596e6ff481777

Description:
The bug is a segmentation fault when closing several secure websockets from inside concurrent python3.6 threads: 
```pythohn
'python3.6 libssl_bug.py' terminated by signal SIGSEGV (Address boundary error)
```

It does not occur when doing it in a single thread, only when closing multiple threads at once.


```python
threads = []
for _ in range(NUM_THREADS):
    t = SocketThread('wss://echo.websocket.org/', ssl_opt={'cert_reqs': 0})
    t.start()
    threads.append(t)

sleep(4)

for t in threads:
    t.keep_running = False
    t.ws.close()  # libssl segfaults on python3.6 when closing a wss:// connection with cert_reqs=0
    t.join()

```

This is my first python bug report, so apologies if I didn't get the formatting right or if I'm missing some info.
msg285973 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-01-22 00:00
I can reproduce the crash under Linux with OpenSSL 1.0.2j and Python 2.7, 3.5 and 3.6. Python 2.7 doesn't crash every time, 1 out of 5 times maybe.

#0  ssl_buf_freelist_free (list=0x7fffe8003b50) at ssl_lib.c:2094
#1  0x00007fffef3d68d7 in SSL_CTX_free (a=0x7fffe8006cf0) at ssl_lib.c:2182
#2  0x00007fffed23c60b in context_dealloc (self=0x7fffecffa9e8) at /home/heimes/dev/python/3.6/Modules/_ssl.c:2787
#3  0x00000000004c8b74 in subtype_dealloc (self=0x7fffecffa9e8) at Objects/typeobject.c:1222
#4  0x00000000004a4bc7 in dict_dealloc (mp=0x7fffed00a480) at Objects/dictobject.c:2011
#5  0x00000000004c8be2 in subtype_dealloc (self=0x7fffecfffce0) at Objects/typeobject.c:1207
msg285974 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-01-22 00:09
SSL_CTX_free:

    if (a->rbuf_freelist)
        ssl_buf_freelist_free(a->rbuf_freelist);

(gdb) p list
$1 = (SSL3_BUF_FREELIST *) 0x7fffe8003b50
(gdb) p *list
$2 = {chunklen = 33096, len = 1, head = 0x7fffe8031c00}
(gdb) p *list->head
$3 = {next = 0x5000030317000000}
(gdb) p *list->head->next
Cannot access memory at address 0x5000030317000000
msg285976 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-01-22 00:35
I can't reproduce the crash with OpenSSL 1.1.0 and LibreSSL 2.5.0. You might have found a bug in OpenSSL.
msg285977 - (view) Author: Nick Sweeting (pirate) * Date: 2017-01-22 00:38
Actually I suspected it was OpenSSL first, I filed the report on their github issues, then went on a fun little wild goose chase that ended in the CPython issue tracker. :)

https://github.com/openssl/openssl/issues/2260

Thanks for helping debug this so quickly!
msg294300 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-05-24 00:13
It's a bug in OpenSSL. There isn't much we can do about it. I'm closing the issue as 3rd party issue.
History
Date User Action Args
2017-05-24 00:13:55christian.heimessetstatus: open -> closed
resolution: third party
messages: + msg294300

stage: resolved
2017-01-22 00:38:57piratesetmessages: + msg285977
2017-01-22 00:35:06christian.heimessetmessages: + msg285976
2017-01-22 00:09:23christian.heimessetmessages: + msg285974
2017-01-22 00:00:49christian.heimessetmessages: + msg285973
2017-01-21 23:37:20piratecreate