This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Fix a BrokenPipeError when a multiprocessing.Queue is garbage collected
Type: crash Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: maggyero
Priority: normal Keywords: patch

Created on 2022-03-15 18:28 by maggyero, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 31913 open maggyero, 2022-03-15 18:28
Messages (3)
msg415272 - (view) Author: Géry (maggyero) * Date: 2022-03-15 18:28
A `BrokenPipeError` exception is raised when the queue thread of a `multiprocessing.Queue` still sends enqueued items to the write end of the queue pipe *after* the read end of the queue pipe has been [automatically closed during its garbage collection](https://docs.python.org/3/library/socket.html#socket.socket.close) following the garbage collection of the queue (the write end of the queue pipe is not garbage collected because it is also referenced by the queue thread):

```
import multiprocessing

def main():
    q = multiprocessing.Queue()
    q.put(0)

if __name__ == '__main__':
    main()
```
msg415273 - (view) Author: Géry (maggyero) * Date: 2022-03-15 18:43
I have attached the following patch: pass a reference to the reader end of the queue pipe to the queue thread so that the reader end is not garbage collected and closed before the queue thread has sent all the buffered data to the writer end.
msg415351 - (view) Author: Géry (maggyero) * Date: 2022-03-16 16:27
I forgot to include the output of the above program:

```
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/queues.py", line 251, in _feed
    send_bytes(obj)
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 205, in send_bytes 
    self._send_bytes(m[offset:offset + size])
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 416, in _send_bytes 
    self._send(header + buf)
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 373, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
```
History
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91185
2022-03-16 16:27:25maggyerosetmessages: + msg415351
2022-03-15 18:43:00maggyerosetmessages: + msg415273
2022-03-15 18:28:50maggyerosetkeywords: + patch
stage: patch review
pull_requests: + pull_request30006
2022-03-15 18:28:33maggyerocreate