classification
Title: Queue with maxsize can lead to deadlocks
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: davin, ned.deily, pitrou, tomMoral
Priority: normal Keywords: patch

Created on 2018-03-15 06:36 by tomMoral, last changed 2018-03-21 21:27 by ned.deily. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6119 merged tomMoral, 2018-03-15 06:42
PR 6178 merged miss-islington, 2018-03-21 15:51
PR 6180 merged tomMoral, 2018-03-21 17:01
Messages (7)
msg313855 - (view) Author: Thomas Moreau (tomMoral) * Date: 2018-03-15 06:36
The fix for the Queue._feeder does not properly handle the size of the Queue. This can lead to a situation where the Queue is considered as Full when it is empty. Here is a reproducing script:

```
import multiprocessing as mp

q = mp.Queue(1)


class FailPickle():
    def __reduce__(self):
        raise ValueError()

q.put(FailPickle())
print("Queue is full:", q.full())
q.put(0)
print(f"Got result: {q.get()}")
```
msg314208 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-21 15:50
New changeset e2f33add635df4fde81be9960bab367e010c19bf by Antoine Pitrou (Thomas Moreau) in branch 'master':
bpo-33078 - Fix queue size on pickling error (GH-6119)
https://github.com/python/cpython/commit/e2f33add635df4fde81be9960bab367e010c19bf
msg314210 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-21 16:21
New changeset bb5b5291971c104ea773db1a30e46d410b6b3e1e by Antoine Pitrou (Miss Islington (bot)) in branch '3.7':
bpo-33078 - Fix queue size on pickling error (GH-6119) (GH-6178)
https://github.com/python/cpython/commit/bb5b5291971c104ea773db1a30e46d410b6b3e1e
msg314211 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-03-21 16:36
Buildbots are failing on macOS:

http://buildbot.python.org/all/#/builders/14/builds/804/steps/4/logs/stdio
http://buildbot.python.org/all/#/builders/93/builds/459/steps/4/logs/stdio
msg314219 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-03-21 18:58
PR 6180 appears to have fixed the 3.x failures, thanks!
msg314221 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-21 19:01
Good, I think we can close again now!
msg314224 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-03-21 21:27
(Fot the record, PR 6181 was the 3.7 backport of PR 6180.)
History
Date User Action Args
2018-03-21 21:27:09ned.deilysetmessages: + msg314224
2018-03-21 19:01:57pitrousetstatus: open -> closed
resolution: fixed
messages: + msg314221

stage: patch review -> resolved
2018-03-21 18:58:34ned.deilysetmessages: + msg314219
2018-03-21 17:01:55tomMoralsetstage: needs patch -> patch review
pull_requests: + pull_request5929
2018-03-21 16:36:17ned.deilysetstatus: closed -> open

nosy: + ned.deily
messages: + msg314211

resolution: fixed -> (no value)
stage: resolved -> needs patch
2018-03-21 16:21:33pitrousetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-03-21 16:21:19pitrousetmessages: + msg314210
2018-03-21 15:51:02miss-islingtonsetpull_requests: + pull_request5928
2018-03-21 15:50:52pitrousetversions: - Python 2.7, Python 3.5, Python 3.6
2018-03-21 15:50:35pitrousetmessages: + msg314208
2018-03-15 06:42:27tomMoralsetkeywords: + patch
stage: patch review
pull_requests: + pull_request5883
2018-03-15 06:36:54tomMoralcreate