feature: extension module C++ howto/example in extending-embedding/c-api
documentation


why:	The embedding/extension documentation states that module
implementation in c++ is possible, without providing any guidance beyond
this. Coders more familiar/comfortable with c++ than c, writing c++ code
to expose in python, might want to create the actual python class/module
implementations in c++ classes or structs. A basic guide can help
prevent wastage of energy reinventing the wheel, and also serve as a
guide to safe/preferred style. The method outlined here can be a
starting point in finding out what that style is. 
Also it seems, to my eyes at least, a little easier to 'visualise' and
apply than the equivalent in plain C (the Noddy module etc), a bit more
'systematic', and easier to understand the relation between c and python
objects. So after some trial and error, discovering the pitfalls, I am
finding it easy to create useful python classes with this recipe. Others
might also find it useful, even if only as a stimulus to find a Better Way.

Python is an object-oriented language noted for its clarity, so an
allegedly simple and clear OO implementation strategy for module classes
should be examined.


what:	I've written a module currently called 'cpeepee', containing a
basic python class ,(struct TestRealDestructor in C++,
'cpeepee.destructo' in python). It has been tested with python-2.5 and
g++-4.2 on ubuntu-'hardy heron' and python-2.5/g++-3.4 on FreeBSD-6.4. 

The c++ struct inherits from PyObject.(You could also inherit from
PyVarObject or other more specialised types). The most important
non-feature of this class is that it is not virtual, has no virtual
destructor or functions(in member objects virtual destructors and the
rest are OK however). Therefore the packing of ob_refcnt and friends is
correct, and casting to/from PyObject is safe(-and otherwise is not -
the PyObject* cast from such a type with virtual destructor is offset by
the size of a pointer -the vptr?-, but when python casts back to the
type - from void* or so?- the offset is not removed, leading to mayhem).

[You could also not inherit, simply put the PyObject_HEAD macro as the
first entry in the struct - could be a class also, as long as the
PyObject members(ob_refcnt, ob_type ... )were public - You would have to
write a new macro to fill in those members in the constructor's
initialiser list, but that doesn't look too hard. As it is the inherited
classes use a function and some macros (almost identical to
PyObject_HEAD_INIT(typo) ) to fill in the PyObject parent. Again, vfuncs
are out]

	The destructo method and member tables are static members of
TestRealDestructor, as is its type object.(Other optional tables etc
should also be static members if provided - makes for a simple
consistent setup)
	The objects constructor is called from
TestRealDestructor::type.tp_new() and passed the args and kwds arguments
it is passed, using placement new with memory obtained from
tp_alloc()(see TestRealDestructor::create() ). Being able to properly
call an object's constructor was the real motivation for writing the
code this derives from. Using C style one is stuck casting a char* to
your type and then filling its fields by hook or crook.
 
On error, either you could throw a c++ exception in constructor to catch
in tp_new, and convert to a python exception, or simply throw the python
exception in the constructor, and then check if PyErr_Occurred() in
tp_new() - the approach with destructo.

Since tp_new and the constructor take care of object creation, this
leaves tp_init not doing much at all. It could be used for extra checks,
or printing funky messages about preserved ham

Functions to be exposed in python API as class member functions should
static members of the class, and to do the work they call ordinary
member functions of the class object passed into the static function .
You could use global static functions(with the first arg
TestRealDestructor* say, rather than PyObject*), but that's less clear,
less systematic, less OO.

Everything can be placed in a namespace to avoid any pollution of global
namespace.

I've worked out the bugs that were obvious to me, so it should compile
and run as is, without error messages. The one compile warning I don't
know how to banish comes from the offsetof macro when setting member
offsets in the member table(as copied from the c example) - however the
object whose offset is so established works fine from python, so I think
it's a spurious warning. 

If you find any issues with my approach, I'm happy to work through them,
or if you know what to do then make whatever changes you think required.

I'm happy to put any code or words I contribute on this topic to go
under python's copyright as long as I'm credited somehow(however you
normally do that). Whatever, I'm happy to contribute to such a great
project as python

sincerely

John O'Driscoll

Did you take a look at Boost.Python?
This is a much more natural (from C++ point of view) way to create
Python types. All boilerplate code comes from template techniques
(creation/destruction, static methods...)

The tutorial is interesting:
http://www.boost.org/doc/libs/release/libs/python/doc/tutorial/doc/html/python/exposing.html

IOW, I suggest to just add a reference to the boost::python library

I'm aware of Boost without being familiar. I should find out more. I 
don't have any reason to think it might not be the better approach.

I guess when I wrote this I was thinking in terms of minimising 
dependencies: writing a program that depended only on the standard 
libraries of c/c++ and python. In that context, if Boost-python were to 
become a part of a stdlib, there'd be no need at all for this. Are there 
any plans afoot?

Also, if dependencies are not a problem for your project, Boost might be 
the way to go. I won't indulge in any polemic one way or the other. I 
just thought a DIY primer might be useful in some contexts. Whether the 
official docs is the place for it I don't know.

John O'Driscoll

I suggest you put your example on the Python wiki or even Python cookbook site. Or announce on Python list and you should get some feedback from C++ users. I am closing this until there is some.

Writing it against 3.x capi would be more useful in the long run if not now.

Most reviewers prefer plain text attachments they can open in the browser.