classification
Title: CPython 3.8.4 regression on __setattr__ in multiinheritance with metaclasses
Type: behavior Stage: patch review
Components: Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: David Caro, christian.heimes, davidism, doko, gvanrossum, jssfr, kam193, lukasz.langa, miss-islington, scoder
Priority: critical Keywords: 3.8regression, 3.9regression, patch

Created on 2020-07-14 14:57 by kam193, last changed 2020-07-20 17:19 by lukasz.langa.

Files
File name Uploaded Description Edit
bpo41295.py christian.heimes, 2020-07-14 15:23
Pull Requests
URL Status Linked Edit
PR 21473 closed David Caro, 2020-07-14 16:29
PR 21528 merged scoder, 2020-07-18 07:45
PR 21540 merged miss-islington, 2020-07-18 21:20
PR 21541 merged miss-islington, 2020-07-18 21:20
Messages (19)
msg373637 - (view) Author: (kam193) Date: 2020-07-14 14:57
CPython 3.8.4 broke previously correct custom __setattr__ implementation, when metaclass inheritances from multiple classes, including not-meta.

Following code:

```
class Meta(type):
    def __setattr__(cls, key, value):
        type.__setattr__(cls, key, value)

class OtherClass:
    pass


class DefaultMeta(OtherClass, Meta):
    pass

obj = DefaultMeta('A', (object,), {})
obj.test = True
print(obj.test)
```

Works in Python up to 3.8.3, but in 3.8.4 it raises:

Traceback (most recent call last):
  File "repr.py", line 13, in <module>
    obj.test = True
  File "repr.py", line 3, in __setattr__
    type.__setattr__(cls, key, value)
TypeError: can't apply this __setattr__ to DefaultMeta object

This change affects e.g. https://github.com/pallets/flask-sqlalchemy/issues/852
msg373640 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-07-14 15:17
The error message is coming from hackcheck function in Objects/typeobject.c, https://github.com/python/cpython/blob/b4cd77de05e5bbaa6a4be90f710b787e0790c36f/Objects/typeobject.c#L5811-L5845 . The issue could be related to bpo-39960. Checking now...
msg373641 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-07-14 15:23
The regression was introduced in commit 8912c182455de83e27d5c120639ec91b18247913

$ git checkout v3.8.4
$ ./configure -C
$ make
$ $ ./python bpo41295.py 
Traceback (most recent call last):
  File "bpo41295.py", line 14, in <module>
    obj.test = True
  File "bpo41295.py", line 3, in __setattr__
    type.__setattr__(cls, key, value)
TypeError: can't apply this __setattr__ to DefaultMeta object

$ git revert 8912c182455de83e27d5c120639ec91b18247913
[detached HEAD 8d3c7847a0] Revert "bpo-39960: Allow heap types in the "Carlo Verre" hack check that override "tp_setattro()" (GH-21092) (GH-21339)"
 3 files changed, 11 insertions(+), 117 deletions(-)
$ make
$ ./python bpo41295.py 
True
msg373642 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-07-14 15:23
Stefan, please take a look.
msg373644 - (view) Author: David Lord (davidism) Date: 2020-07-14 15:41
It appears to be solved in Flask-SQLAlchemy's development version already, as the mixins now inherit from `type`. We caught the issue when we started applying flake8 (possibly through flake8-bugbear).
msg373646 - (view) Author: David Caro (David Caro) * Date: 2020-07-14 16:32
Just sent a PR that fixes the issue (a first approach), let me know if it's addressing the issue correctly or if there's a better way.


Thanks!
msg373766 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2020-07-16 19:58
David, which issue number is this?
msg373803 - (view) Author: David Caro (David Caro) * Date: 2020-07-17 08:31
> David, which issue number is this?

It's issue41295, pr #21473
msg373897 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2020-07-18 12:32
PR 21528 works for all four test cases that we now have (1x test_capi.py, 3x test_descr.py).

Any comments? We need to merge a fix before Monday to meet the deadline of the planned hotfix release.

@kam193, could you please also test that change with your real code?
msg373920 - (view) Author: miss-islington (miss-islington) Date: 2020-07-18 21:19
New changeset c53b310e5926266ce267c44a168165cacd786d6e by scoder in branch 'master':
bpo-41295: Reimplement the Carlo Verre "hackcheck" (GH-21528)
https://github.com/python/cpython/commit/c53b310e5926266ce267c44a168165cacd786d6e
msg373921 - (view) Author: miss-islington (miss-islington) Date: 2020-07-18 21:37
New changeset 38d930f2ccbff6f93c4c54a7a6a1759266136504 by Miss Islington (bot) in branch '3.8':
bpo-41295: Reimplement the Carlo Verre "hackcheck" (GH-21528)
https://github.com/python/cpython/commit/38d930f2ccbff6f93c4c54a7a6a1759266136504
msg373922 - (view) Author: miss-islington (miss-islington) Date: 2020-07-18 21:39
New changeset 01ab9634601fc1a4f9ac5d72ddc022239d2543fe by Miss Islington (bot) in branch '3.9':
bpo-41295: Reimplement the Carlo Verre "hackcheck" (GH-21528)
https://github.com/python/cpython/commit/01ab9634601fc1a4f9ac5d72ddc022239d2543fe
msg373923 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-07-18 22:06
We have a buildbot failure: test_asyncio altered the execution environment. What does that mean? Victor?
msg373924 - (view) Author: (kam193) Date: 2020-07-18 23:35
@Stefan: Today I run unit tests for Flask-SQLAlchemy (v2.4.3) as well as for aioxmpp (another library reported this problem https://github.com/horazont/aioxmpp/issues/342). In both cases patch is successful: tests fail on Python 3.8.4 but pass on patched Python.

As for me it looks done. Thanks for everyone involved!
msg373934 - (view) Author: Jonas Schäfer (jssfr) Date: 2020-07-19 06:41
@kam193 Thanks for running the aioxmpp tests. I built the patched python yesterday, but I didn’t manage to get a virtualenv with it up&running.

Good to hear that the fix works as expected!
msg373936 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2020-07-19 06:57
> test_asyncio altered the execution environment

That happened several times before in previous builds. I think there's just a part of the asyncio tests that is unreliable. I left a comment in issue 41273 since it might be related, the sporadic failures started appearing in the build following the merge.
msg373965 - (view) Author: (kam193) Date: 2020-07-19 14:13
@Jonas: I know, running those tests isn't so easy, I think a few libraries are incompatible with python 3.9/3.10 yet or I didn't find right prerequisites. But fortunately, everything is running on patched 3.8, so after back-porting I can run it
msg373967 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-07-19 15:15
@Łukasz: this should be good for the 3.8.5 release.
msg374018 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-07-20 17:19
Released.
History
Date User Action Args
2020-07-20 17:19:24lukasz.langasetpriority: release blocker -> critical

messages: + msg374018
2020-07-19 15:15:24gvanrossumsetmessages: + msg373967
2020-07-19 14:13:58kam193setmessages: + msg373965
2020-07-19 06:57:55scodersetmessages: + msg373936
2020-07-19 06:41:14jssfrsetmessages: + msg373934
2020-07-18 23:35:01kam193setmessages: + msg373924
2020-07-18 22:06:36gvanrossumsetnosy: + gvanrossum
messages: + msg373923
2020-07-18 21:39:06miss-islingtonsetmessages: + msg373922
2020-07-18 21:37:49miss-islingtonsetmessages: + msg373921
2020-07-18 21:20:18miss-islingtonsetpull_requests: + pull_request20683
2020-07-18 21:20:08miss-islingtonsetpull_requests: + pull_request20682
2020-07-18 21:19:57miss-islingtonsetnosy: + miss-islington
messages: + msg373920
2020-07-18 12:32:21scodersetmessages: + msg373897
2020-07-18 07:45:53scodersetpull_requests: + pull_request20664
2020-07-17 08:31:23David Carosetmessages: + msg373803
2020-07-16 19:58:11dokosetnosy: + doko
messages: + msg373766
2020-07-15 13:59:52jssfrsetnosy: + jssfr
2020-07-14 16:32:27David Carosetmessages: + msg373646
2020-07-14 16:29:23David Carosetkeywords: + patch
nosy: + David Caro

pull_requests: + pull_request20617
stage: patch review
2020-07-14 15:41:44davidismsetnosy: + davidism
messages: + msg373644
2020-07-14 15:23:55christian.heimessetnosy: + scoder
messages: + msg373642
2020-07-14 15:23:17christian.heimessetfiles: + bpo41295.py
priority: normal -> release blocker
versions: + Python 3.9, Python 3.10
messages: + msg373641

keywords: + 3.9regression
2020-07-14 15:17:19christian.heimessetnosy: + christian.heimes
messages: + msg373640
2020-07-14 15:13:49christian.heimessetkeywords: + 3.8regression
nosy: + lukasz.langa
2020-07-14 14:57:16kam193create