This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Performance of typing._ProtocolMeta._get_protocol_attrs and isinstance
Type: performance Stage:
Components: Library (Lib) Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: levkivskyi, orenbenkiki
Priority: normal Keywords:

Created on 2017-05-29 09:16 by orenbenkiki, last changed 2022-04-11 14:58 by admin.

Messages (2)
msg294686 - (view) Author: Oren Ben-Kiki (orenbenkiki) Date: 2017-05-29 09:16
In 3.6.0, invocation of isinstance calls typing._ProtocolMeta._get_protocol_attrs.
This creates a set of all attributes in all base classes, loops on these attributes to check they exist, and discards the set. It is very slow.

My program uses isinstance to allow for flexibility in parameter types in certain key functions. I realize that using isinstance is frowned upon, but it seems to make sense in my case.

As a result, >95% of its run-time is inside typing._ProtocolMeta._get_protocol_attrs (!).

I have created a simple wrapper around isinstance which caches its result with a Dict[Tuple[type, type], bool]. This solved the performance problem, but introduced a different problem - type checking.

I use mypy and type annotations, and my code cleanly type-checks (with the occasional # type: ignore). If I switch to using my own isinstance function, then mypy's type inference no longer treats it as special, so it starts complaining about all uses of values protected by if isinstance(value, SomeType): ...

I propose that either the private typing._ProtocolMeta.__subclasscheck__ (which invokes _get_protocol_attrs), or the public isinstance, would be modified to cache their results.

I can create a PR for either approach, if this is acceptable.
msg295044 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-06-02 17:44
Thanks for reporting!

The runtime implementation of protocol classes will be thoroughly reworked as a part of PEP 544, see also https://github.com/python/typing/pull/417 for a proof of concept runtime implementation.

Also, there is another ongoing discussion https://github.com/python/typing/issues/432 about a global refactoring of typing module that will significantly improve performance.

Therefore, I would wait with any large PRs until these two stories are settled. If you still want to propose a small PR, you can do this at the upstream typing repo https://github.com/python/typing
History
Date User Action Args
2022-04-11 14:58:47adminsetgithub: 74690
2017-06-02 17:44:59levkivskyisetnosy: + levkivskyi
messages: + msg295044
2017-05-29 09:16:49orenbenkikicreate