Author njs
Recipients Mark.Shannon, benjamin.peterson, larry, njs, pitrou, serhiy.storchaka
Date 2015-08-30.07:55:12
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1440921314.81.0.101345933913.issue24912@psf.upfronthosting.co.za>
In-reply-to
Content
Python goes to great lengths to make __class__ assignment work in general (I was surprised too). Historically the exception has been that instances of statically allocated C extension type objects could not have their __class__ reassigned. Semantically, this is totally arbitrary, and Guido even suggested that fixing this would be a good idea in this thread:
   https://mail.python.org/pipermail/python-dev/2014-December/137430.html
The actual rule that's causing problems here is that *immutable* objects should not allow __class__ reassignment. (And specifically, objects which are assumed to be immutable by the interpreter. Most semantically immutable types in Python are defined by users and their immutability falls under the "consenting adults" rule; it's not enforced by the interpreter. Also this is very different from "hashable" -- even immutability isn't a requirement for being hashable, e.g., all user-defined types are mutable and hashable by default!)

By accident this category of "immutable-by-interpreter-invariant" has tended to be a subset of "defined in C using the old class definition API", so fixing the one issue uncovered the other.

This goal that motivated this patch was getting __class__ assignment to work on modules, which are mutable and uncacheable and totally safe. With this patch it becomes possible to e.g. issue deprecation warnings on module attribute access, which lets us finally deprecate some horrible global constants in numpy that have been confusing users for a decade now:
   http://mail.scipy.org/pipermail/numpy-discussion/2015-July/073205.html
   https://pypi.python.org/pypi/metamodule

I'd *really* prefer not to revert this totally, given the above. (Also, if you read that python-dev message above, you'll notice that the patch also caught and fixed a different really obscure interpreter-crashing bug.)

I think the Correct solution would be to disallow __class__ assignment specifically for the classes where it violates invariants.

If this is considered to be release critical -- and I'm not entirely sure it is, given that similar tricks have long been possible and are even referenced in the official docs?
  https://www.reddit.com/r/Python/comments/2441cv/can_you_change_the_value_of_1/ch3dwxt
  https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong
-- but if it is considered to be release critical, and it's considered too short notice to accomplish a proper fix, then a simple hack would be to add something like

if (!(oldto->tp_flags & Py_TPFLAGS_HEAPTYPE) && oldto != &PyModule_Type) {
    PyErr_Format(PyExc_TypeError, ...);
    return -1;
}

to typeobject.c:object_set_class. As compared to reverting the whole patch, this would preserve the most important case, which is one that we know is safe, and we could then progressively relax the check further in the future...
History
Date User Action Args
2015-08-30 07:55:14njssetrecipients: + njs, pitrou, larry, benjamin.peterson, Mark.Shannon, serhiy.storchaka
2015-08-30 07:55:14njssetmessageid: <1440921314.81.0.101345933913.issue24912@psf.upfronthosting.co.za>
2015-08-30 07:55:14njslinkissue24912 messages
2015-08-30 07:55:12njscreate