classification
Title: `_scproxy` calls SystemConfiguration functions in a way that can cause deadlocks
Type: crash Stage: resolved
Components: macOS Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Maxime Belanger, benjamin.peterson, miss-islington, ned.deily, ronaldoussoren
Priority: normal Keywords: patch

Created on 2017-10-30 19:38 by Maxime Belanger, last changed 2018-09-12 15:38 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4178 merged maxbelanger, 2017-10-30 19:59
PR 9215 open miss-islington, 2018-09-12 15:20
PR 9216 merged miss-islington, 2018-09-12 15:20
Messages (6)
msg305247 - (view) Author: Maxime Belanger (Maxime Belanger) Date: 2017-10-30 19:38
Through the use of various Python packages (such as `pyobjc`), it is possible for a deadlock to occur due to how `_scproxy.c` calls `SCDynamicStoreCopyProxies`.

In more recent versions of macOS (10.7+), this function can block on acquiring a lock deep inside `NSUserPreferences`. As `pyobjc` allows Python-wrapped `NSString`s to be stored in `CFPreferences`, it is thus possible for one thread to hold the `CFPreferences` lock and block on the GIL while another thread holds the GIL and blocks on the `CFPreferences` lock.

We've observed this on a significant number of macOS devices before fixing ourselves by wrapping the calls to `SCDynamicStoreCopyProxies` with `Py_BEGIN/END_ALLOW_THREADS`.
msg325090 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2018-09-11 23:14
New changeset 4859ba0d2ce4506fddc3f55f90f8dce031b3804f by Benjamin Peterson (Max Bélanger) in branch 'master':
closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178)
https://github.com/python/cpython/commit/4859ba0d2ce4506fddc3f55f90f8dce031b3804f
msg325104 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-09-12 00:21
Thanks, Benjamin!

Ronald, do you think this should be backported?
msg325135 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2018-09-12 12:33
Yes it should, the same problem has been present from the start.
msg325140 - (view) Author: miss-islington (miss-islington) Date: 2018-09-12 15:37
New changeset 72c34cf6dd5fb206fe4d407ff603468af466fd2e by Miss Islington (bot) in branch '3.7':
closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178)
https://github.com/python/cpython/commit/72c34cf6dd5fb206fe4d407ff603468af466fd2e
msg325141 - (view) Author: miss-islington (miss-islington) Date: 2018-09-12 15:38
New changeset aa12534acff4868f6dce9e2364a66fbb0ebcb3ca by Miss Islington (bot) in branch '3.6':
closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178)
https://github.com/python/cpython/commit/aa12534acff4868f6dce9e2364a66fbb0ebcb3ca
History
Date User Action Args
2018-09-12 15:38:39miss-islingtonsetmessages: + msg325141
2018-09-12 15:37:43miss-islingtonsetnosy: + miss-islington
messages: + msg325140
2018-09-12 15:20:47miss-islingtonsetpull_requests: + pull_request8648
2018-09-12 15:20:37miss-islingtonsetpull_requests: + pull_request8647
2018-09-12 12:33:27ronaldoussorensetmessages: + msg325135
2018-09-12 00:21:04ned.deilysetmessages: + msg325104
2018-09-11 23:14:03benjamin.petersonsetstatus: open -> closed

nosy: + benjamin.peterson
messages: + msg325090

resolution: fixed
stage: patch review -> resolved
2017-10-30 19:59:15maxbelangersetkeywords: + patch
stage: patch review
pull_requests: + pull_request4148
2017-10-30 19:38:53Maxime Belangercreate