This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: [C API] Test that the Python C API is compatible with C++
Type: enhancement Stage: test needed
Components: C API, Tests Versions: Python 3.11
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gregory.p.smith, vstinner
Priority: normal Keywords:

Created on 2022-03-30 14:14 by vstinner, last changed 2022-04-11 14:59 by admin.

Messages (2)
msg416359 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-30 14:14
There are more and more popular projects using the Python C API. The first big player is pybind11:
"Seamless operability between C++11 and Python"

Recently, I proposed a PR to add Python 3.11 support to the datatable project:

My PR uses pythoncapi_compat.h header file which provides recent C API functions to old Python functions. The header file implements these functions as static inline function.

Problem: a static inline function implemented in a C header file used in a C++ code file can emit C++ compiler warnings.

In datatable, I got two kinds of C++ compiler warnings:

* Usage of the C NULL constant: C++ prefers nullptr
* "Old-style" cast like (PyObject*)obj: C++ prefers static_cast, reinterpret_cast, etc.

It seems like these compiler warnings are not enabled by default. The datatable project seems enabling them in its CI and I was asked to fix these warnings.

In the pythoncapi-compat project (*), I chose to use nullptr and reinterpret_cast if the "__cplusplus" macro is defined. Example:

// C++ compatibility
#ifdef __cplusplus
#  define PYCAPI_COMPAT_CAST(TYPE, EXPR) reinterpret_cast<TYPE>(EXPR)
#  define PYCAPI_COMPAT_NULL nullptr

// Cast argument to PyObject* type.
#ifndef _PyObject_CAST
#  define _PyObject_CAST(op) PYCAPI_COMPAT_CAST(PyObject*, op)


It's unclear to me if the Python C API has or has not the same issue than pythoncapi_compat.h.

Last years, some old macros of the Python C API have been converted to static inline functions, like Py_INCREF(). It's unclear to me if these compiler warnings happen on Py_INCREF(). I don't understand why, but static inline macros from Python.h didn't emit compiler warnings in datatable, whereas similar static inline functions of pythoncapi_compat.h emitted compiler warnings.

Maybe there is a difference between <Python.h> and "Python.h". Or maybe it depends if the header file is a "local" file, or a "system" header file (ex: installed in /usr/include/ on Linux).

A first step would be to build a C++ extension as part of the Python test suite and check that there is no compiler warning. My GH-32175 PR is a proof-of-concept of that.

I don't know which C++ version we should target. pybind11 targets C++11. See bpo-39355 for a discussion about C++20: usage of the C++20 "module" keyword... which is a "contextual keyword" in practice.
msg416756 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2022-04-05 06:49
If we can conditionally test new things based on C++XX version, accumulating modern issue regression tests seems useful. Otherwise 11 at minimum.

As for why some things trigger this and others don't, my wild _guess_ would be whether the statements appear within an extern "C" block. Though that's not really what that is for so it isn't clear to me.
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91321
2022-04-05 06:49:28gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg416756

components: + Tests
stage: test needed
2022-03-30 14:14:35vstinnercreate