classification
Title: Add helpers to populate modules in C
Type: enhancement Stage: patch review
Components: C API Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, erlendaasland, serhiy.storchaka, shihai1991
Priority: normal Keywords: patch

Created on 2020-11-16 19:27 by christian.heimes, last changed 2021-11-16 17:23 by erlendaasland.

Pull Requests
URL Status Linked Edit
PR 23286 open christian.heimes, 2020-11-16 19:31
Messages (1)
msg381144 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-11-16 19:27
It's currently inconvenient and sometimes error-prone to initialize module members in C. Even stdlib modules are sometimes missing error checks or have reference counting issues in error path. I propose add three helpers to deal with common cases:

1) declaration of simple attributes (int, str, bool, float) in an array of structs
2) creation and addition of a new type from a type spec
3) creation and addition a new exception object


(1) Simple attribute declaration uses a NULL terminated array of PyModuleConst_Def. The internal definition of the type can be hidden by macros. Example:

   static PyModuleConst_Def example_constants[] = {
       PyModuleConst_None("none_value"),
       PyModuleConst_Long("integer", 42),
       PyModuleConst_Bool("false_value", 0),
       PyModuleConst_Bool("true_value", 1),
       PyModuleConst_String("somestring", "Hello"),
       PyModuleConst_LongMacro(EXAMPLE_INT),
       PyModuleConst_StringMacro(EXAMPLE_STRING),
       {NULL},
   }

A new function "int PyModule_AddConstants(PyObject *module, PyModuleConst_Def *def)" populates the module object with attributes.


(2) Type initialization

The function "PyTypeObject * PyModule_AddNewTypeFromSpec(PyObject *module, PyType_Spec *spec, PyObject *base)" is PyType_FromModuleAndSpec() + PyModule_AddType(). Additionally to PyType_FromModuleAndSpec() it also handles "base" argument with single type instance. The extra feature avoids "PyTuple_Pack(1, baseclass)" in the caller. The function adds a strong reference to the module object and returns a borrowed reference. The borrowed reference is designed module state assignment. 


(3) exception creation

The function "PyObject* PyModule_AddNewException(PyObject *module, const char *name, const char *doc, PyObject *base, PyObject *dict)" creates a new exception, calculates the attribute name for the qualified exception name, and adds the exception to the module. It's PyErr_NewExceptionWithDoc() + PyModule_AddObjectRef().


Note about (1): I first proposed to add PyModuleDef.m_constants, however we cannot extend the struct easily becaues it conflicts with stable ABI. Petr mentioned that he has plans to deal with the issue. Also see discussion https://discuss.python.org/t/define-module-constants-in-pymoduledef/5749
History
Date User Action Args
2021-11-16 17:23:51erlendaaslandsetnosy: + erlendaasland
2020-11-26 04:31:16shihai1991setnosy: + shihai1991
2020-11-23 10:44:06serhiy.storchakasetnosy: + serhiy.storchaka
2020-11-16 19:31:34christian.heimessetkeywords: + patch
pull_requests: + pull_request22213
2020-11-16 19:27:21christian.heimescreate