classification
Title: GenericPyCData_new does not invoke new or init
Type: behavior Stage:
Components: ctypes Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: LewisGaul, aeros, b1tninja, beccamorgan
Priority: normal Keywords:

Created on 2019-11-20 02:34 by b1tninja, last changed 2020-02-19 04:46 by aeros.

Files
File name Uploaded Description Edit
ctypesnew.py b1tninja, 2019-11-20 02:34 Example of Structure subclass that is instantiated without new or init
Messages (7)
msg357022 - (view) Author: Justin Capella (b1tninja) * Date: 2019-11-20 02:34
When subclassing the ctypes.Structure class, __new__ and __init__ are not invoked when using the inherited classmethod from_buffer_copy to create the object.

I think this is because tp_alloc is ultimately used by GenericPyCData_new when creating the object using the from_buffer_copy classmethod inherited from _CData.

https://github.com/python/cpython/blob/be143ec99674ba38c5811f34cdb85ef39c2dc8f8/Modules/_ctypes/_ctypes.c#L3202

Expected behavior: creation of Structure subclass object would invoke __new__ and possibly __init__.
msg358223 - (view) Author: Rebecca Morgan (beccamorgan) * Date: 2019-12-10 20:42
Hi there, I’m a newcomer would like to take a look at this. Planning to investigate over the next few days.
msg358265 - (view) Author: Rebecca Morgan (beccamorgan) * Date: 2019-12-11 15:25
Looking at the test case 'test_from_buffer_copy' here:
https://github.com/python/cpython/blob/master/Lib/ctypes/test/test_frombuffer.py

This suggests the failure to call init in this instance is expected behavior (similarly for from_buffer), unlike e.g. the classmethod fromkeys() with a dictionary subclass which behaves as expected.

Are there any required changes here, perhaps if not code then documentation?
msg358276 - (view) Author: Justin Capella (b1tninja) * Date: 2019-12-11 18:31
It might make sense that CSimpleData types do not call init, which makes sense for _fields_ but it doesn't make sense to offer subclassing (of Structure) and not use the subclass, instead creating a type of the same name.

That test case is bugged, missing an underscore... The git blame wasn't "available" for that particular test but I would argue that it's testing for undocumented behavior :p

I wanted to be able to do assertions/initialization, I'm sure other people have expected from_buffer(_copy) et al would act as a class method, returning an instance of the subclass.

More generally, I think that GenericPyCData_new should not "tp_alloc"... What's _new about that?

My guess is there was at some level desire to "optimize" ctypes. If that does turn out to be the case I'd hope for a "unsimple" CData and ask that Structure be an instance of that.

Either way docs reflecting the behavior to Structure would be nice, I just hope it includes "prior to version" :p thanks for taking a look
msg360452 - (view) Author: Justin Capella (b1tninja) * Date: 2020-01-22 08:27
Becca were you still interested in this issue? 

Anyone agree the unit test seems to have a bug if that is intended behavior?

Is there a better forum for discussion about design/behavior, maybe the more generic issue of GenericPyCData_new using tp_alloc instead of tp_new
msg362210 - (view) Author: Rebecca Morgan (beccamorgan) * Date: 2020-02-18 13:45
I would still be interested in this, with some clarity on the testing/expected behavior.
msg362248 - (view) Author: Kyle Stanley (aeros) * (Python committer) Date: 2020-02-19 04:46
Justin Capella wrote:
> Is there a better forum for discussion about design/behavior, maybe the more generic issue of GenericPyCData_new using tp_alloc instead of tp_new

Rebecca Morgan wrote:
> I would still be interested in this, with some clarity on the testing/expected behavior.

I'm not personally active on the capi-sig mailing list, but it seems like it would be a suitable location for discussing the behavior of GenericPyCData_new. If you don't receive a response in capi-sig, you could also try python-dev to bring some attention to the issue since most of the core developers actively check there.
History
Date User Action Args
2020-02-19 04:46:34aerossetmessages: + msg362248
2020-02-18 13:45:02beccamorgansetmessages: + msg362210
2020-01-22 08:27:19b1tninjasetmessages: + msg360452
2019-12-11 18:31:59b1tninjasetmessages: + msg358276
2019-12-11 15:25:09beccamorgansetmessages: + msg358265
2019-12-11 14:58:40LewisGaulsetnosy: + aeros, LewisGaul
2019-12-10 20:42:15beccamorgansetnosy: + beccamorgan
messages: + msg358223
2019-11-20 02:34:04b1tninjacreate