classification
Title: Race condition in asyncore may access the wrong dispatcher
Type: resource usage Stage:
Components: Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Nir Soffer, walkhour
Priority: normal Keywords:

Created on 2017-07-14 16:44 by walkhour, last changed 2017-07-21 00:07 by haypo.

Pull Requests
URL Status Linked Edit
PR 2707 open walkhour, 2017-07-14 16:48
PR 2764 open walkhour, 2017-07-19 15:54
Messages (3)
msg298359 - (view) Author: Jaume (walkhour) * Date: 2017-07-14 16:44
If a socket is closed and polled here https://github.com/python/cpython/blob/0d0a32fb91cdfea1626e6c6b77a9bc44e15a2b8a/Lib/asyncore.py#L183 the flag returned will be select.POLLNVAL (note that this can happen despite dispatcher.close() being called).

This wouldn't be a problem but before here https://github.com/python/cpython/blob/0d0a32fb91cdfea1626e6c6b77a9bc44e15a2b8a/Lib/asyncore.py#L185 a new socket may be created with the same fd (which wouldn't be strange since that fd is now available), so the retrieved socket will be the newly created one instead of the old one and the new one will be close.

This is regularly happening to us, I could try to explain better, but that's really it.
msg298362 - (view) Author: Nir Soffer (Nir Soffer) Date: 2017-07-14 17:17
Can you provide a minimal reproducer, or best add a failing test?
msg298682 - (view) Author: Nir Soffer (Nir Soffer) Date: 2017-07-19 16:21
Adding more info after discussion in github.

After polling readable/writeable dispatchers, asyncore.poll(2) receive a list of ready file descriptors, and invoke callbacks on the dispatcher objects.

If a dispatchers is closed and and a new dispatcher is created, the new dispatcher may get the same file descriptor. If the file descriptor was in the ready list returned from poll()/select(), asyncore may try to invoke one of the callbacks (e.g. handle_write) on the new dispatcher.

Here is an example in asycore.poll()

        r, w, e = select.select(r, w, e, timeout)

        for fd in r:
            obj = map.get(fd)
            if obj is None:
                continue
            read(obj)

read close obj, removing fd from map, then creates a new one
getting the same fd...

        for fd in w:
            obj = map.get(fd)

this get the new object from the map, instead of the closed one.

            if obj is None:
                continue
            write(obj)

invoke write on the wrong socket, which is not writable

        for fd in e:
            obj = map.get(fd)

same issue here

            if obj is None:
                continue
            _exception(obj)


asyncore.poll2() has same issue:

        r = pollster.poll(timeout)
        for fd, flags in r:
            obj = map.get(fd)
            if obj is None:
                continue
            readwrite(obj, flags)

fd may have been closed and recreated by in a previous iteration of the loop.

This issue is demonstrated in the failing test:
https://github.com/python/cpython/pull/2707/commits/5aeb0098d2347984f3a89cf036c305edd2b8382b
History
Date User Action Args
2017-07-21 00:07:26hayposetversions: - Python 3.3, Python 3.4
2017-07-19 16:21:53Nir Soffersetmessages: + msg298682
title: Race condition in asyncore wrongly closes channel -> Race condition in asyncore may access the wrong dispatcher
2017-07-19 15:54:35walkhoursetpull_requests: + pull_request2823
2017-07-14 17:17:04Nir Soffersetnosy: + Nir Soffer
messages: + msg298362
2017-07-14 16:48:25walkhoursetpull_requests: + pull_request2771
2017-07-14 16:44:47walkhoursettype: resource usage
versions: + Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
2017-07-14 16:44:08walkhourcreate