classification
Title: PyType_FromSpec*() overwrites the type's "__module__"
Type: behavior Stage: resolved
Components: C API Versions: Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, ncoghlan, petr.viktorin, scoder, serhiy.storchaka, shihai1991
Priority: normal Keywords: patch

Created on 2020-05-20 21:24 by scoder, last changed 2020-10-20 11:48 by petr.viktorin.

Pull Requests
URL Status Linked Edit
PR 20273 merged scoder, 2020-05-20 21:37
PR 20782 merged miss-islington, 2020-06-10 16:09
Messages (4)
msg369476 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2020-05-20 21:24
The PyType_FromSpec() functions set the type's "__module__" attribute at the end:

https://github.com/python/cpython/blob/0509c4547fc95cc32a91ac446a26192c3bfdf157/Objects/typeobject.c#L3154-L3172

There are only two possible cases, either it finds a module name in the spec name and sets "__module__" from that, or it outputs a deprecation warning. Both behaviours are annoying because they ignore anything that the type already has in its dict, e.g. a property entry from the "members" or "getset" structs.

Since this code can't really be moved before the call to "PyType_Ready()" (which creates the type's dict and populates it), I think the best fix would be to first check if "__module__" is already in the dict and only otherwise take care of setting it.

I noticed this when trying to make the "__module__" attribute of Cython's coroutine type work with type specs, which should actually return the instance specific module name, i.e. the module that defines the coroutine, not the module that defines the type. This would normally be solved with an entry in "members", but that is discarded by the code above. While this approach means that the type does not have its own "__module__" entry, I think the use cases of pickling and documentation support it, because they care about the module name of the instance, not of the type.

I think working around this behaviour is somewhat easy by creating a new descriptor with PyDescr_NewMember() and re-adding it after calling PyType_FromSpec*(), so this is something that can just be fixed in Py3.9+.

Relevant tickets: issue 15146 for setting "__module__" and issue 20204 for the deprecation warning.
msg371215 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2020-06-10 16:09
New changeset 24b8bad6d30ae4fb37ee686a073adfa5308659f9 by scoder in branch 'master':
bpo-40703: Let PyType_FromSpec() set "type.__module__" only if it is not set yet. (GH-20273)
https://github.com/python/cpython/commit/24b8bad6d30ae4fb37ee686a073adfa5308659f9
msg371220 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2020-06-10 16:43
New changeset 9419158a3e67ba2eadf33215568003ed723b0a98 by Miss Islington (bot) in branch '3.9':
bpo-40703: Let PyType_FromSpec() set "type.__module__" only if it is not set yet. (GH-20273) (GH-20782)
https://github.com/python/cpython/commit/9419158a3e67ba2eadf33215568003ed723b0a98
msg379114 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2020-10-20 11:48
Thank you, Stefan!

This should have a test as well.
I'm willing to mentor someone who wants to get into the C-API, otherwise this has low priority for me.
History
Date User Action Args
2020-10-20 11:48:55petr.viktorinsetstatus: closed -> open
resolution: fixed ->
2020-10-20 11:48:48petr.viktorinsetstatus: open -> closed
resolution: fixed
messages: + msg379114

stage: test needed -> resolved
2020-06-10 16:43:59scodersetstage: patch review -> test needed
2020-06-10 16:43:16scodersetmessages: + msg371220
2020-06-10 16:09:15miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request19977
2020-06-10 16:09:09scodersetmessages: + msg371215
2020-05-21 05:03:18shihai1991setnosy: + shihai1991
2020-05-20 21:37:44scodersetkeywords: + patch
stage: patch review
pull_requests: + pull_request19553
2020-05-20 21:24:32scodercreate