This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing Queue empty() is broken on Windows
Type: Stage: resolved
Components: Library (Lib), Windows Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Radosław.Szkodziński, eryksun, sbt, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2014-12-02 10:09 by Radosław.Szkodziński, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
testmp.py Radosław.Szkodziński, 2014-12-02 10:09 Testcase - hangs
Messages (2)
msg231984 - (view) Author: Radosław Szkodziński (Radosław.Szkodziński) Date: 2014-12-02 10:09
multiprocessing.Queue.empty() uses pipe polling on Windows. Unfortunately, pipe semantics on windows are different from POSIX.

The result is either:
- Hang when one process tries to get() and another checks for empty()
- Falsely returning empty() == False despite nothing being possible to get - because the other process is actually trying to read from the queue.

The attached testcase demonstrates the first case, with main process hard hanging on my Python 2.7.8 (anaconda), 32-bit on Windows 7 64-bit. Whether 1 or 2 happens depends on specific program flow.
Both can cause deadlocks in code that should be valid.

Note that get(block=False) actually works correctly.

It is possible that the problem is also present in Python 3.
msg232056 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-12-02 22:53
This also hangs for me in 2.7.8 64-bit, Windows 7. To poll the pipe, the parent process calls PeekNamedPipe, which blocks because the child has already called ReadFile.

> It is possible that the problem is also present in Python 3.

multiprocessing switched to overlapped I/O in 3.3, which avoids this problem.
History
Date User Action Args
2022-04-11 14:58:10adminsetgithub: 67165
2021-02-23 11:30:24eryksunsetstatus: open -> closed
resolution: out of date
stage: resolved
2014-12-02 22:53:07eryksunsetnosy: + eryksun
messages: + msg232056
2014-12-02 15:40:56r.david.murraysetnosy: + sbt
2014-12-02 10:09:33Radosław.Szkodzińskicreate