Title: Blake2b/s implementations have minor GIL issues
Type: behavior Stage:
Components: Extension Modules Versions: Python 3.8, Python 3.7, Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, gwk
Priority: normal Keywords:

Created on 2019-04-03 15:04 by gwk, last changed 2019-04-03 15:20 by christian.heimes.

Messages (2)
msg339394 - (view) Author: George King (gwk) * Date: 2019-04-03 15:04
I was browsing the Blake2b module implementation in master and noticed two subtle issues in blake2b_impl.c. There are two places where the GIL gets released; both of them appear flawed.

py_blake2b_new_impl, line 221. The ALLOW_THREADS block fails to acquire/release self->lock.

_blake2_blake2b_update, line 279. The lock is lazily allocated correctly on line 279. However the test on 282 that chooses to release the GIL or not fails to take into account the length test. This means that once a large block is fed to `update`, then every call to update will release the GIL, even if it is a single byte.

It should look something more like this:
bool should_allow_threads = (buf.len >= HASHLIB_GIL_MINSIZE);
if (should_allow_threads && self->lock == NULL)
  self->lock = PyThread_allocate_lock();
if (should_allow_threads && self->lock != NULL) { ... }
else { ... }

This respects the size criterion, and also protects against the case where the lock allocation fails.
msg339396 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-04-03 15:20
Thanks, I'll have a look.
Date User Action Args
2019-04-03 15:20:57christian.heimessetassignee: christian.heimes
messages: + msg339396
versions: + Python 3.6, Python 3.8
2019-04-03 15:10:40SilentGhostsetnosy: + christian.heimes
type: behavior
2019-04-03 15:04:35gwkcreate