Title: PyType_FromSpecWithBases tp_new bugfix
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.5
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Dormouse759, Robin.Schreiber, belopolsky, jcea, loewis, ncoghlan, petr.viktorin, pitrou, serhiy.storchaka
Priority: normal Keywords: patch, pep3121

Created on 2012-08-19 15:57 by Robin.Schreiber, last changed 2018-09-11 20:55 by petr.viktorin.

File name Uploaded Description Edit
PyType_FromSpecWithBases_tp_new_fix.patch Robin.Schreiber, 2012-08-19 15:57
PyType_FromSpecWithBases_tp_new_fix_v1.patch Robin.Schreiber, 2012-11-30 18:01
Messages (6)
msg168579 - (view) Author: Robin Schreiber (Robin.Schreiber) * (Python triager) Date: 2012-08-19 15:57
As with every type, that has been created and initialized, HeapTypes created form PyType_FromSpecWithBases() have to pass through PyType_Ready(). Here the function inherit_special might be called, which, among other things, does the following:
3892         if (base != &PyBaseObject_Type ||                                                                                                                
3893             (type->tp_flags & Py_TPFLAGS_HEAPTYPE)) {
3894             if (type->tp_new == NULL)
3895                 type->tp_new = base->tp_new;
3896         }
The code does not know of Heaptypes that have been created from extension-types by the PEP 384 refactorings. This includes extension-types that do not specify a tp_new method but instead have seperate factory methods, that are only used within the extension module. inherit_special() might consequently assign inappropriate new-methods to these type objects.

To circumvent this issue, I propose to enhance PyType_FromSpecWithBases in the following way: If no tp_new has been specified, we assign the newly defined PySpec_New() method to tp_new which simply denies the user to create an instance of this type. This also prohibits inherit_special to falsely inherit inappropriate new-methods.
msg170157 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-09-10 03:17
Can you give an example of the situation that you described?  Perhaps you encountered it while refactoring some particular extension module.  Which?

In your patch new code is commented out.

PySpec_New() is not a good name.  It should be something like PyObject_FailingNew().
msg176695 - (view) Author: Robin Schreiber (Robin.Schreiber) * (Python triager) Date: 2012-11-30 18:01
Here is revised version of the patch. 

Martin von Löwis and I had discovered a way to reproduce problems with refactored modules, that
occur without this patch. This is was several months ago, however I will try to give you a code sample! :-)
msg176805 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-12-02 18:40
Robin, do you remember which extension types were affected by this issue?
msg277880 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-02 10:21
Affected modules were tkinter and curses (see issue23815). This was fixed by explicit nullifying tp_new after calling PyType_FromSpec(). Issue26979 was open for documenting this danger effect.
msg325059 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2018-09-11 20:55
Converting static types to heap ones is just one of the reasons to use PyType_FromSpec*. Another ones are writing such classes from scratch, and converting pure-Python classes to C. I don't think PyObject_FailingNew is a good default for either of those.

I think Sehriy's solution in bpo-26979 is better: document this, and allow setting a slot to NULL explicitly.
Date User Action Args
2018-09-11 20:55:58petr.viktorinsetmessages: + msg325059
2018-04-09 10:48:45petr.viktorinsetnosy: + Dormouse759
2018-04-08 11:06:49ncoghlansetnosy: + ncoghlan, petr.viktorin
2016-10-02 10:22:24serhiy.storchakasettype: behavior -> crash
versions: + Python 3.5, Python 3.6, Python 3.7, - Python 3.2, Python 3.3, Python 3.4
2016-10-02 10:21:38serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg277880
2012-12-02 18:40:32pitrousetnosy: + pitrou

messages: + msg176805
versions: + Python 3.2, Python 3.3
2012-11-30 18:01:34Robin.Schreibersetfiles: + PyType_FromSpecWithBases_tp_new_fix_v1.patch
keywords: + patch
messages: + msg176695
2012-11-08 13:21:13Robin.Schreibersetkeywords: + pep3121, - patch
2012-09-10 03:17:59belopolskysetnosy: + belopolsky
messages: + msg170157
2012-09-10 02:32:08jceasetnosy: + jcea
2012-08-19 15:57:45Robin.Schreibercreate