New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Acquiring locks not interrupted by signals on musl libc #78185
Comments
When Python is built on Alpine Linux or in any other configuration that uses musl libc, calls to Lock.acquire() can't be interrupted by signals. This bug is caught by test_lock_acquire_interruption and test_rlock_acquire_interruption in 3.6/Lib/test/test_threadsignals.py, both of which fail when Python is built on musl. POSIX explicitly says that sem_timedwait ever failing with EINTR is optional: And musl deliberately chooses not to support it: However, we incorrectly rely on it for correct operation. Our documentation https://docs.python.org/3.6/library/threading.html#threading.Lock.acquire just says that "Lock acquires can now be interrupted by signals on POSIX", not that any optional POSIX features are required for it. A similar bug was bpo-11223, which was the same problem but with pthread_cond_timedwait, which is used when semaphores aren't available. |
How is this considered "fixed"? Why couldn't this be actually fixed by using eventfd instead of semaphores when they're available, for example? |
Reimplementing locks with eventfd can be another issue. Such a change can only land in a new Python version, though. We'll just have to consider musl unsupported for interrupting locks in our current maintenance releases as I have done. How likely is it that musl will change to allow returning EINTR from sem_timedwait given that the only stated reason not to is very old kernel versions? |
Re musl changing their behavior, see https://www.openwall.com/lists/musl/2018/09/07/1 and the resulting thread. In addition to the old kernel version issue, two other issues were raised:
|
On Wed, Sep 12, 2018, at 16:03, Joseph Sible wrote:
The race is a good point. However, switching to eventfd is not trivial. It would be the third locking implementation in python just for pthreads. Using eventfd would require reimplementing all the userspace parts of locking (atomics, appropriate memory barriers, and spinning) ourselves. futex is carefully optimized for userspace locking, whereas I've never heard of normal program locking being done with eventfd before. If the expected usecase for interrupting locks is user pressing C-c, being 99.99999% correct is probably enough. So, it's fine with me if you open another issue for reimplementing locks with eventfd. But, I don't personally have the desire to do it. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: