classification
Title: poll()/epoll() are not thread-safe
Type: crash Stage: needs patch
Components: Extension Modules Versions: Python 3.2, Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: select.poll is not thread safe
View: 8865
Assigned To: Nosy List: christian.heimes, neologix, serhiy.storchaka
Priority: normal Keywords:

Created on 2013-01-11 08:36 by neologix, last changed 2013-01-11 09:00 by neologix. This issue is now closed.

Files
File name Uploaded Description Edit
test.py neologix, 2013-01-11 08:36
Messages (3)
msg179647 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2013-01-11 08:36
After optimizing epoll() to use a per-instance buffer like poll() does (http://bugs.python.org/issue16876), I realized that it wasn't thread-safe, and can result in crashes:
"""
 ./python /tmp/test.py
*** glibc detected *** ./python: free(): corrupted unsorted chunks: 0x0000000001c8e690 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3041c75676]
./python[0x4fd9ae]
./python(PyThreadState_DeleteCurrent+0x10c)[0x4fdb91]
./python[0x52a444]
/lib64/libpthread.so.0[0x30428077e1]
/lib64/libc.so.6(clone+0x6d)[0x3041ce153d]
======= Memory map: ========
[...]
"""

The problem is that poll (and now epoll) array is stored per-instance, and if second call to poll() is made while another thread is currently blocked on poll(), the array is reallocated (through PyMem_Resize()) if the buffer size has changed (if the number of FDs registered has changed).

"""
static int
update_ufd_array(pollObject *self)
[...]
    self->ufd_len = PyDict_Size(self->dict);
    PyMem_RESIZE(self->ufds, struct pollfd, self->ufd_len);
[...]
"""

can be called while another thread is blocked there:
"""
    /* call poll() */
    Py_BEGIN_ALLOW_THREADS
    poll_result = poll(self->ufds, self->ufd_len, timeout);
    Py_END_ALLOW_THREADS
"""

So when the first call to poll() returns, it can end up in SIGSEGV or heap corruption (unless the array was increased an realloc() was able to expand it in-place).

Reproducer script attached.
msg179653 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-11 08:50
For poll() see issue8865.
msg179656 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2013-01-11 08:58
OK, I'll close as duplicate.
History
Date User Action Args
2013-01-11 09:00:48neologixsetstatus: open -> closed
2013-01-11 08:58:36neologixsetsuperseder: select.poll is not thread safe
dependencies: - select.poll is not thread safe
resolution: duplicate
messages: + msg179656
2013-01-11 08:52:15serhiy.storchakasetdependencies: + select.poll is not thread safe
2013-01-11 08:50:48serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg179653
2013-01-11 08:36:44neologixcreate