Message355029
After upgrading from Python 3.7.3 to Python 3.7.4, we noticed an increased runtime of about 50% in some tests of our application.
After some testing we constructed the following hypothesis: Accessing instance attributes of classes that define a custom metaclass take more time in Python 3.7.4. The access takes the more time the more classes are in the MRO of the concerning class (in other words the more base classes the concerning class has got).
We verified the hypothesis by creating a minimal example (file attached) that produced the output at the bottom. (The index defines the number of base classes. The four variants are with/without a metaclass and with simple/multiple inheritance. The kind of inheritance has no influence on the runtime.)
In Python 3.7.4 (in contrast to 3.7.3 and 3.8), you can see that the usage of a metaclass results in a linear increase of the runtime of attribute accesses depending on the number of base classes.
We could also identify the issue/commit that introduced the performance issue:
https://bugs.python.org/issue28866
https://github.com/python/cpython/commit/bfd0b7720196b9ff647cc33dafbd31a04496402c
In the Python 3.8 branch, the performance issue seems to be resolved by:
https://bugs.python.org/issue36922
https://github.com/python/cpython/commit/988fff5d0e7fccecbf776c08ec56695820b3b4a8
Will this performance issue be addressed in a future 3.7 version (e.g. 3.7.6) or is it in your optinion sufficient that the issue seems to be resolved in 3.8?
Python 3.7.3 (constant runtime)
3.668036e-08 noMetaBaseX[1].attr
3.630355e-08 noMetaBaseX[2].attr
3.687350e-08 noMetaBaseX[5].attr
3.827102e-08 noMetaBaseX[10].attr
3.724956e-08 noMetaBaseX[20].attr
3.912778e-08 noMetaBaseX[50].attr
4.072552e-08 noMetaBaseX[100].attr
3.620875e-08 metaBaseX[1].attr
3.640054e-08 metaBaseX[2].attr
3.599339e-08 metaBaseX[5].attr
3.606570e-08 metaBaseX[10].attr
3.946514e-08 metaBaseX[20].attr
3.715085e-08 metaBaseX[50].attr
3.762364e-08 metaBaseX[100].attr
3.667802e-08 noMetaRecBaseX[1].attr
3.579415e-08 noMetaRecBaseX[2].attr
3.647997e-08 noMetaRecBaseX[5].attr
3.577428e-08 noMetaRecBaseX[10].attr
3.686319e-08 noMetaRecBaseX[20].attr
3.832261e-08 noMetaRecBaseX[50].attr
4.087625e-08 noMetaRecBaseX[100].attr
3.654509e-08 noMetaRecBaseX[1].attr
3.617825e-08 noMetaRecBaseX[2].attr
3.573970e-08 noMetaRecBaseX[5].attr
3.664559e-08 noMetaRecBaseX[10].attr
3.702526e-08 noMetaRecBaseX[20].attr
3.774067e-08 noMetaRecBaseX[50].attr
3.836915e-08 noMetaRecBaseX[100].attr
Python 3.7.4 (see lines marked with ->)
4.042378e-08 noMetaBaseX[1].attr
3.704071e-08 noMetaBaseX[2].attr
3.868766e-08 noMetaBaseX[5].attr
3.716869e-08 noMetaBaseX[10].attr
3.693100e-08 noMetaBaseX[20].attr
3.947849e-08 noMetaBaseX[50].attr
3.790146e-08 noMetaBaseX[100].attr
-> 5.010019e-08 metaBaseX[1].attr
-> 5.215336e-08 metaBaseX[2].attr
-> 6.103393e-08 metaBaseX[5].attr
-> 7.947347e-08 metaBaseX[10].attr
-> 1.082155e-07 metaBaseX[20].attr
-> 2.264330e-07 metaBaseX[50].attr
-> 4.951039e-07 metaBaseX[100].attr
3.727378e-08 noMetaRecBaseX[1].attr
3.815971e-08 noMetaRecBaseX[2].attr
3.661390e-08 noMetaRecBaseX[5].attr
3.709824e-08 noMetaRecBaseX[10].attr
3.785757e-08 noMetaRecBaseX[20].attr
3.813983e-08 noMetaRecBaseX[50].attr
3.856352e-08 noMetaRecBaseX[100].attr
-> 5.260115e-08 metaRecBaseX[1].attr
-> 5.661043e-08 metaRecBaseX[2].attr
-> 6.760995e-08 metaRecBaseX[5].attr
-> 8.038124e-08 metaRecBaseX[10].attr
-> 1.100538e-07 metaRecBaseX[20].attr
-> 2.246786e-07 metaRecBaseX[50].attr
-> 4.877147e-07 metaRecBaseX[100].attr
Python 3.8.0rc1 (constant runtime)
4.045787e-08 noMetaBaseX[1].attr
4.206206e-08 noMetaBaseX[2].attr
4.209608e-08 noMetaBaseX[5].attr
4.049552e-08 noMetaBaseX[10].attr
4.133343e-08 noMetaBaseX[20].attr
4.229599e-08 noMetaBaseX[50].attr
4.259420e-08 noMetaBaseX[100].attr
4.146996e-08 metaBaseX[1].attr
4.140459e-08 metaBaseX[2].attr
4.143048e-08 metaBaseX[5].attr
4.054639e-08 metaBaseX[10].attr
4.219960e-08 metaBaseX[20].attr
4.222810e-08 metaBaseX[50].attr
4.420123e-08 metaBaseX[100].attr
4.044410e-08 noMetaRecBaseX[1].attr
4.043790e-08 noMetaRecBaseX[2].attr
4.209061e-08 noMetaRecBaseX[5].attr
4.042041e-08 noMetaRecBaseX[10].attr
4.048202e-08 noMetaRecBaseX[20].attr
4.260607e-08 noMetaRecBaseX[50].attr
4.267928e-08 noMetaRecBaseX[100].attr
4.037958e-08 metaRecBaseX[1].attr
3.968827e-08 metaRecBaseX[2].attr
4.011758e-08 metaRecBaseX[5].attr
3.954960e-08 metaRecBaseX[10].attr
4.057979e-08 metaRecBaseX[20].attr
4.145640e-08 metaRecBaseX[50].attr
4.148783e-08 metaRecBaseX[100].attr |
|
Date |
User |
Action |
Args |
2019-10-21 08:15:44 | ttrd | set | recipients:
+ ttrd |
2019-10-21 08:15:44 | ttrd | set | messageid: <1571645744.43.0.285485842632.issue38541@roundup.psfhosted.org> |
2019-10-21 08:15:44 | ttrd | link | issue38541 messages |
2019-10-21 08:15:43 | ttrd | create | |
|