classification
Title: OpenSSL 1.1.1: use SSL_write_ex() and SSL_read_ex()
Type: enhancement Stage: resolved
Components: SSL Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: FFY00, christian.heimes, ethan.furman, geofft, miss-islington, pablogsal
Priority: high Keywords: easy (C), patch

Created on 2021-01-07 08:23 by christian.heimes, last changed 2021-07-23 15:26 by pablogsal. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 25468 merged christian.heimes, 2021-04-19 03:55
PR 27271 merged pablogsal, 2021-07-20 20:52
PR 27308 merged miss-islington, 2021-07-23 15:06
Messages (13)
msg384567 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-01-07 08:23
SSL_read() and SSL_write() are limited to int. The new SSL_write_ex() and SSL_read_ex() APIs support size_t just like read(2) and recv(2). Also see bpo-42853.

int SSL_write_ex(SSL *s, const void *buf, size_t num, size_t *written);
int SSL_read_ex(SSL *ssl, void *buf, size_t num, size_t *readbytes);

Both functions return 1 for success or 0 for failure.
msg384568 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-01-07 08:35
As of version 3.3.1, LibreSSL does not have SSL_write_ex() and SSL_read_ex(). The read and write functions are limited to int.
msg391295 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-17 18:14
3.10 branch now requires OpenSSL 1.1.1. This should be easy to implement.
msg391358 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-19 04:03
SSL_write_ex() and SSL_read_ex() solve two issues:

* bpo-42853: SSLSocket no longer raises overflow error when sending or receiving more than 2 GB of data
* bpo-31711: empty send(b"") no longer fails with protocol violation exception
msg391363 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-19 04:55
New changeset 89d1550d14ba689af12eeb726e4ff8ce73cee7e1 by Christian Heimes in branch 'master':
bpo-42854: Use SSL_read/write_ex() (GH-25468)
https://github.com/python/cpython/commit/89d1550d14ba689af12eeb726e4ff8ce73cee7e1
msg391422 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-20 06:30
Ethan, what's your platform and OpenSSL version?
msg391521 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-04-21 13:55
False alarm, sorry.  Still getting used to merging, rebasing, etc.

Current tests run fine.
msg397822 - (view) Author: Geoffrey Thomas (geofft) * Date: 2021-07-19 17:56
I am still seeing failures to read responses over 2 GB in Python 3.10b1. I'm working on a reproducer, but I'm getting the same "OverflowError: signed integer is greater than maximum" that I get in 3.9.
msg397832 - (view) Author: Geoffrey Thomas (geofft) * Date: 2021-07-19 18:34
Christian mentioned on Twitter that this is probably due to a missing argument clinic change from "int" to "Py_ssize_t". I can confirm that fixing that and rerunning argument clinic makes things start to work.

I don't have the ability to reopen this bug (I think), can someone reopen it please?
msg397833 - (view) Author: Filipe LaĆ­ns (FFY00) * (Python triager) Date: 2021-07-19 18:40
https://twitter.com/geofft/status/1417167982665551877
msg397906 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-20 20:39
I will push a fix today
msg398063 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-23 15:06
New changeset 83d1430ee5b8008631e7f2a75447e740eed065c1 by Pablo Galindo Salgado in branch 'main':
bpo-42854: Correctly use size_t for _ssl._SSLSocket.read and _ssl._SSLSocket.write (GH-27271)
https://github.com/python/cpython/commit/83d1430ee5b8008631e7f2a75447e740eed065c1
msg398068 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-23 15:26
New changeset 5ec275758dbc307e9838e7038bfc3b5390950ea7 by Miss Islington (bot) in branch '3.10':
bpo-42854: Correctly use size_t for _ssl._SSLSocket.read and _ssl._SSLSocket.write (GH-27271) (GH-27308)
https://github.com/python/cpython/commit/5ec275758dbc307e9838e7038bfc3b5390950ea7
History
Date User Action Args
2021-07-23 15:26:03pablogsalsetmessages: + msg398068
2021-07-23 15:26:03pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-23 15:06:05miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25853
2021-07-23 15:06:02pablogsalsetmessages: + msg398063
2021-07-20 20:52:14pablogsalsetstage: needs patch -> patch review
pull_requests: + pull_request25815
2021-07-20 20:39:25pablogsalsetmessages: + msg397906
2021-07-20 19:41:52ned.deilysetnosy: + pablogsal

stage: resolved -> needs patch
2021-07-19 18:40:14FFY00setstatus: closed -> open

nosy: + FFY00
messages: + msg397833

resolution: fixed -> (no value)
2021-07-19 18:34:52geofftsetmessages: + msg397832
2021-07-19 17:56:02geofftsetnosy: + geofft
messages: + msg397822
2021-04-21 13:55:39ethan.furmansetstatus: open -> closed
resolution: fixed
messages: + msg391521
2021-04-21 13:54:05ethan.furmansetmessages: - msg391419
2021-04-20 06:30:22christian.heimessetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg391422
2021-04-20 01:01:59ethan.furmansetnosy: + ethan.furman
messages: + msg391419
2021-04-19 04:55:50christian.heimessetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-04-19 04:55:36christian.heimessetmessages: + msg391363
2021-04-19 04:03:40christian.heimessetmessages: + msg391358
2021-04-19 04:01:15christian.heimeslinkissue31711 superseder
2021-04-19 03:55:43christian.heimessetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request24193
2021-04-17 18:14:02christian.heimessetkeywords: + easy (C)
priority: normal -> high
messages: + msg391295

stage: needs patch
2021-01-07 08:35:16christian.heimessetmessages: + msg384568
2021-01-07 08:23:09christian.heimescreate