Author jhaberman
Recipients Alexander.Belopolsky, Arfrever, Robin.Schreiber, amaury.forgeotdarc, belopolsky, jcea, jhaberman, lekma, loewis, pitrou
Date 2021-07-28.21:45:00
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1627508700.92.0.663681174459.issue15870@roundup.psfhosted.org>
In-reply-to
Content
I know this is quite an old bug that was closed almost 10 years ago.  But I am wishing this had been accepted; it would have been quite useful for my case.

I'm working on a new iteration of the protobuf extension for Python.  At runtime we create types dynamically, one for each message defined in a .proto file, eg. from "message Foo" we dynamically construct a "class Foo".

I need to support class variables like Foo.BAR_FIELD_NUMBER, but I don't want to put all these class variables into tp_dict because there are a lot of them and they are rarely used.  So I want to implement __getattr__ for the class, which requires having a metaclass.  This is where the proposed PyType_FromSpecEx() would have come in very handy.

The existing protobuf extension gets around this by directly calling PyType_Type.tp_new() to create a type with a given metaclass:

https://github.com/protocolbuffers/protobuf/blob/53365065d9b8549a5c7b7ef1e7e0fd22926dbd07/python/google/protobuf/pyext/message.cc#L278-L279

It's unclear to me if PyType_Type.tp_new() is intended to be a supported/public API.  But in any case, it's not available in the limited API, and I am trying to restrict myself to the limited API.  (I also can't use PyType_GetSlot(PyType_Type, Py_tp_new) because PyType_Type is not a heap type.)

Put more succinctly, I do not see any way to use a metaclass from the limited C API.

Possible solutions I see:

1. Add PyType_FromSpecEx() (or similar with a better name) to allow a metaclass to be specified.  But I want to support back to at least Python 3.6, so even if this were merged today it wouldn't be viable for a while.

2. Use eval from C to create the class with a metaclass, eg.
      class Foo(metaclass=MessageMeta)

3. Manually set FooType->ob_type = &MetaType, as recommended here: https://stackoverflow.com/a/52957978/77070 .  Since PyObject.ob_type is part of the limited API, I think this might be possible!
History
Date User Action Args
2021-07-28 21:45:00jhabermansetrecipients: + jhaberman, loewis, jcea, amaury.forgeotdarc, belopolsky, pitrou, Arfrever, lekma, Alexander.Belopolsky, Robin.Schreiber
2021-07-28 21:45:00jhabermansetmessageid: <1627508700.92.0.663681174459.issue15870@roundup.psfhosted.org>
2021-07-28 21:45:00jhabermanlinkissue15870 messages
2021-07-28 21:45:00jhabermancreate