This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Add helpers to populate modules in C
Type: enhancement Stage: patch review
Components: C API Versions: Python 3.10
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, erlendaasland, serhiy.storchaka, shihai1991
Priority: normal Keywords: patch

Created on 2020-11-16 19:27 by christian.heimes, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 23286 open christian.heimes, 2020-11-16 19:31
Messages (1)
msg381144 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-11-16 19:27
It's currently inconvenient and sometimes error-prone to initialize module members in C. Even stdlib modules are sometimes missing error checks or have reference counting issues in error path. I propose add three helpers to deal with common cases:

1) declaration of simple attributes (int, str, bool, float) in an array of structs
2) creation and addition of a new type from a type spec
3) creation and addition a new exception object

(1) Simple attribute declaration uses a NULL terminated array of PyModuleConst_Def. The internal definition of the type can be hidden by macros. Example:

   static PyModuleConst_Def example_constants[] = {
       PyModuleConst_Long("integer", 42),
       PyModuleConst_Bool("false_value", 0),
       PyModuleConst_Bool("true_value", 1),
       PyModuleConst_String("somestring", "Hello"),

A new function "int PyModule_AddConstants(PyObject *module, PyModuleConst_Def *def)" populates the module object with attributes.

(2) Type initialization

The function "PyTypeObject * PyModule_AddNewTypeFromSpec(PyObject *module, PyType_Spec *spec, PyObject *base)" is PyType_FromModuleAndSpec() + PyModule_AddType(). Additionally to PyType_FromModuleAndSpec() it also handles "base" argument with single type instance. The extra feature avoids "PyTuple_Pack(1, baseclass)" in the caller. The function adds a strong reference to the module object and returns a borrowed reference. The borrowed reference is designed module state assignment. 

(3) exception creation

The function "PyObject* PyModule_AddNewException(PyObject *module, const char *name, const char *doc, PyObject *base, PyObject *dict)" creates a new exception, calculates the attribute name for the qualified exception name, and adds the exception to the module. It's PyErr_NewExceptionWithDoc() + PyModule_AddObjectRef().

Note about (1): I first proposed to add PyModuleDef.m_constants, however we cannot extend the struct easily becaues it conflicts with stable ABI. Petr mentioned that he has plans to deal with the issue. Also see discussion
Date User Action Args
2022-04-11 14:59:38adminsetgithub: 86542
2021-11-16 17:23:51erlendaaslandsetnosy: + erlendaasland
2020-11-26 04:31:16shihai1991setnosy: + shihai1991
2020-11-23 10:44:06serhiy.storchakasetnosy: + serhiy.storchaka
2020-11-16 19:31:34christian.heimessetkeywords: + patch
pull_requests: + pull_request22213
2020-11-16 19:27:21christian.heimescreate