This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Use fewer statics in Argument Clinic.
Type: Stage: needs patch
Components: Argument Clinic Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.snow Nosy List: eric.snow, larry, serhiy.storchaka, ta1hia
Priority: normal Keywords:

Created on 2019-09-12 14:59 by eric.snow, last changed 2022-04-11 14:59 by admin.

Messages (5)
msg352193 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-09-12 14:59
(This is a sub-task of bpo-36876, "Global C variables are a problem.".)

Currently Argument Clinic generates "_PyArg_Parser _parser" as a static variable.  Dropping "static" solves the problem of thread safety (e.g. for subinterpreters not sharing the GIL).
msg352226 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-09-12 19:10
Dropping "static" will hit performance. _PyArg_Parser is used at first place because it can cache some data between calls. If drop static, you should drop also _PyArg_Parser and return to old slow arguments parsing.
msg352306 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-09-13 12:15
Would adding a mutex for thread-safe initialization of _Py_Identifier and _PyArg_Parser solve the problem with subinterpreters?
msg352310 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-09-13 12:36
That might work. :)

There are two key problems under subinterpreters that do not share the GIL:

* races on refcount operations
* cache thrashing due to refcount operations (under multi-core threads)

A lock would certainly mitigate the first problem.  I'm not sure how much the second would actually be a problem.
msg352311 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2019-09-13 12:48
shared objects x threads = contention for notification of invalidated cache lines

If you're not running multiple threads, there's no problem.  If it's only a few shared objects, it probably wouldn't be a big deal.  As they say in medicine: "the dose makes the poison."
History
Date User Action Args
2022-04-11 14:59:20adminsetgithub: 82322
2019-09-26 17:28:25ta1hiasetnosy: + ta1hia
2019-09-13 12:48:28larrysetmessages: + msg352311
2019-09-13 12:36:01eric.snowsetmessages: + msg352310
2019-09-13 12:15:27serhiy.storchakasetmessages: + msg352306
2019-09-12 19:10:24serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg352226
2019-09-12 15:10:02larrysettitle: Use less statics in Argument Clinic. -> Use fewer statics in Argument Clinic.
2019-09-12 14:59:42eric.snowcreate