classification
Title: multiprocessing.Queue.close doesn't behave as documented
Type: Stage: resolved
Components: Library (Lib) Versions:
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: davin, graingert, gvanrossum, pitrou
Priority: normal Keywords:

Created on 2019-04-10 10:42 by graingert, last changed 2020-09-16 19:41 by gvanrossum. This issue is now closed.

Messages (3)
msg339847 - (view) Author: Thomas Grainger (graingert) * Date: 2019-04-10 10:42
The docs for https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue.close read:

> Indicate that no more data will be put on this queue by the current process. The background thread will quit once it has flushed all buffered data to the pipe. This is called automatically when the queue is garbage collected.

From this text it seems to me as though the queue should be used as follows:

import contextlib
import multiprocessing


def worker(q):
    with contextlib.closing(q):
        q.put_nowait('hello')


def controller():
    q = multiprocessing.Queue()
    q.close()  # no more 'put's from this process
    p = multiprocessing.Process(target=worker, args=(q, ))
    p.start()
    assert q.get() == 'hello'
    p.join()
    assert p.exitcode == 0
    print('OK!')


if __name__ == '__main__':
    controller()

however I get this:

Traceback (most recent call last):
  File "controller.py", line 22, in <module>
    controller()
  File "controller.py", line 15, in controller
    assert q.get() == 'hello'
  File "/usr/lib/python3.7/multiprocessing/queues.py", line 94, in get
    res = self._recv_bytes()
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 212, in recv_bytes
    self._check_closed()
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed
msg339849 - (view) Author: Thomas Grainger (graingert) * Date: 2019-04-10 10:43
Should the docs be:

> Indicate that no more data will be put on or got from this queue by the current process. The background thread will quit once it has flushed all buffered data to the pipe. This is called automatically when the queue is garbage collected.

Or should the Queue allow closing only one direction of the bidirectional channel?
msg377018 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-16 19:40
Closing for lack of activity.
History
Date User Action Args
2020-09-16 19:41:07gvanrossumsetstatus: open -> closed
stage: resolved
2020-09-16 19:40:54gvanrossumsetnosy: + gvanrossum
messages: + msg377018
2019-04-10 19:20:20brett.cannonsetnosy: + pitrou, davin
components: + Library (Lib)
2019-04-10 10:43:53graingertsetmessages: + msg339849
2019-04-10 10:42:31graingertcreate