Title: Convert collections (cmp_op, hasconst, hasname and others) in opcode module to more optimal type
Type: performance Stage:
Components: Library (Lib) Versions: Python 3.8, Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: godaygo, larry, serhiy.storchaka
Priority: normal Keywords:

Created on 2018-04-21 18:52 by godaygo, last changed 2018-04-26 21:12 by godaygo.

Messages (4)
msg315576 - (view) Author: Kirill Balunov (godaygo) Date: 2018-04-21 18:52
The opcode module contains several collections:


which are only used for `in` checks. At the same time, they are stored as `list`s and `cmp_op` as tuple. Both these types are not optimal for `__contains__` checks. Maybe it is worth at least to convert them to `frozenset` type after they are filled?
msg315817 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-26 20:06
This list are short. I don't think there is a large benefit of this change. But there is a small risk of breaking a third party code (for example hasjrel+hasjabs).
msg315818 - (view) Author: Kirill Balunov (godaygo) Date: 2018-04-26 20:51
Small risk of breaking is a fair point (maybe some FutureWarning with new __getattr__ PEP 562?). I've checked several packages:

vstinner/bytecode:: uses:

    def _has_jump(opcode):
        return (opcode in _opcode.hasjrel
                or opcode in _opcode.hasjabs)

maynard:: defines them as sets and does not rely on opcode module.

all_jumps = absolute_jumps | relative_jumps

numba:: converts them to frozensets:

JREL_OPS = frozenset(dis.hasjrel)
JABS_OPS = frozenset(dis.hasjabs)

codetransfromer:: uses:

absjmp = opcode in hasjabs
reljmp = opcode in hasjrel

--- uses

elif opcode in hasjrel or opcode in hasjabs:

byteplay:: converts them to set:

hasjrel = set(Opcode(x) for x in opcode.hasjrel)
hasjabs = set(Opcode(x) for x in opcode.hasjabs)
hasjump = hasjrel.union(hasjabs)

byterun:: uses:

elif byteCode in dis.hasjrel:
    arg = f.f_lasti + intArg
elif byteCode in dis.hasjabs:
    arg = intArg

In fact, all of the above indicated does not mean anything, but I have not found cases of hasjrel+hasjabs.

Despite the fact that they are small, on average, with sets I gain 5-6x speed-up.
msg315820 - (view) Author: Kirill Balunov (godaygo) Date: 2018-04-26 21:12
I apologize for FutureWarning and __getattr__. I myself do not understand what I meant and how it will help in this situation :)
Date User Action Args
2018-04-26 21:12:52godaygosetmessages: + msg315820
2018-04-26 20:51:55godaygosetmessages: + msg315818
2018-04-26 20:06:20serhiy.storchakasetmessages: + msg315817
2018-04-26 19:41:51godaygosetnosy: + larry, serhiy.storchaka
2018-04-21 18:52:42godaygocreate