classification
Title: compiler: merge co_code and co_lnotab
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, methane, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2020-10-31 07:50 by methane, last changed 2021-02-10 00:21 by methane. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23056 merged methane, 2020-10-31 07:58
Messages (5)
msg380045 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2020-10-31 07:50
co_consts are merged already, but I forget about code attributes other than co_consts.
Duplication ratio is vary by code style, but in case of logging, we can reduce 25% of them.

```
>>> data = open('Lib/logging/__pycache__/__init__.python-310.pyc', 'rb').read()
>>> import marshal
>>> x = []
>>> def walk(c):
...     x.append(c.co_code)
...     x.append(c.co_lnotab)
...     for cc in c.co_consts:
...         if hasattr(cc, 'co_code'):
...             walk(cc)
...
>>> code = marshal.loads(data[16:])
>>> walk(code)
>>> len(x)
336
>>> len(set(x))
249
>>> 249/336
0.7410714285714286
```
msg380049 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-31 10:16
Do you mean sharing values of co_code and co_lnotab between code objects of the same module?

How much memory does this save (in absolute and relative value)? Which functions have the same co_code?
msg380050 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2020-10-31 10:23
PEP 626 has just been accepted.
Could you please leave this until I have merged in the implementation of PEP 626 which uses a different line number table format
msg380061 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2020-10-31 13:08
@Serhiy 

> Do you mean sharing values of co_code and co_lnotab between code objects of the same module?

Yes.

> How much memory does this save (in absolute and relative value)?

Maybe 1~3%, but I am not sure. I am more interested in reducing number of objects, because it will reduce import time.
Additionally, co_code is used while executing code. Unlike other cold data (e.g. docstring, annotations), sharing co_code will improve CPU cache utilization.

> Which functions have the same co_code?

For example,

```
# logging/__init__.py

    @property
    def manager(self):
        return self.logger.manager
...
    @property
    def name(self):
        return self.logger.name
```

Such simple functions are very common in OO-style code.

@Mark Shannon

Sure.
msg386749 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-02-10 00:21
New changeset bdb941be423bde8b02a5695ccf51c303d6204bed by Inada Naoki in branch 'master':
bpo-42217: compiler: merge same co_code and co_linetable objects (GH-23056)
https://github.com/python/cpython/commit/bdb941be423bde8b02a5695ccf51c303d6204bed
History
Date User Action Args
2021-02-10 00:21:17methanesetmessages: + msg386749
2021-02-10 00:20:59methanesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-10-31 13:08:27methanesetmessages: + msg380061
2020-10-31 10:23:57Mark.Shannonsetnosy: + Mark.Shannon
messages: + msg380050
2020-10-31 10:16:11serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg380049
2020-10-31 07:58:03methanesetkeywords: + patch
stage: patch review
pull_requests: + pull_request21975
2020-10-31 07:50:02methanecreate