I did try the suggestion to return Py_False, but that gives the wrong result since Py_False is not 0 and gets returned as Py_True.

I looked for similar code, and this looks like the convention for handling "if obj".  PyObject_IsTrue() is called on the object.  What it returns can be affected by an nb_bool function that returns 0 or 1 (or -1).

Here are a few similar examples that need to implement nb_bool to handle converting an obj to a bool:
Objects/floatobject.c:float_bool
Modules/_datetimemodule.c:delta_bool
Objects/complexobject.c:complex_bool

For many types, there is no nb_bool, and other things are tried such as getting the length.