classification
Title: Stripping annotations out as a new optimization mode
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Adding a way to strip annotations from compiled bytecode
View: 36466
Assigned To: Nosy List: BTaskaya, eric.smith, pablogsal, rhettinger
Priority: normal Keywords:

Created on 2020-03-26 23:03 by BTaskaya, last changed 2020-11-20 19:45 by rhettinger. This issue is now closed.

Files
File name Uploaded Description Edit
simple_tester.py BTaskaya, 2020-03-26 23:02
Messages (7)
msg365118 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-03-26 23:02
Just like docstrings, annotations do nothing at runtime for the majority of the time. We can just strip out them and gain as much as the docstring optimization in bytecode size on a fully annotated repo.

For comparing these two optimizations, I calculated the bytecode weight (marshal dumped size of) of each optimization (with a similar implementation to the compiler but not exact) over a project which both rich in docstrings and annotations. 

Project: https://github.com/Instagram/LibCST

 $ python simple_tester.py LibCST 
Total bytes: 1820086
Total bytes after 629 docstrings (total length of 180333) removed: 1643315
Total bytes after 8859 type annotations removed: 1641594

(I've submitted the script I used to calculate these results.)
msg365119 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-03-26 23:03
As an addition, I need to clarify that each optimization applied by its own.
msg365121 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-03-27 00:03
Note that dataclasses will break without annotations.
msg365122 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-03-27 00:18
> Note that dataclasses will break without annotations.

Yes, that is also affecting this simple tester script. There is no alternative to value-less annotated assignment.

I don't think this is preferable but just for information, these are the results for keeping AnnAssign nodes but just removing their annotation

    def visit_AnnAssign(self, node):
        self.stats += 1
        node.annotation = ast.copy_location(ast.Constant(value=None), node.annotation)
        return node

Total bytes: 1820086
Total bytes after 629 docstrings (total length of 180333) removed: 1643315
Total bytes after 8859 type annotations removed: 1664649

This way we can still support dataclasses but still gain as close as before
msg365124 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-03-27 00:49
I am -1 to this feature because compared with other optimization levels this can have unknown effects on the runtime, especially on dependencies you do not control. dataclasses is an example, but much more exist. To support this feature we would need also another extension for pyc files (like pyo) but (pyo) cannot be used to preserve backwards compatibility. Also, the feature will mainly serve to reduce file size, not much to speed up the runtime as any gain will be constant at definition time. IMHO the maintenance cost is too high for what the value is and it also has some potentially unintended dangerous consequences.
msg365130 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-03-27 02:48
Duplicate of https://bugs.python.org/issue3646

This was rejected because it breaks functools.singledispatch(), typing.NamedTuple(), and dataclasses.dataclass().  Also the space savings was negligible -- typically take much less space than for docstrings because the annotations dict consists of pointers to existing objects.
msg381501 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-11-20 19:45
Corrected link:  https://bugs.python.org/issue36466
History
Date User Action Args
2020-11-20 19:45:43rhettingersetsuperseder: Adding a way to strip annotations from compiled bytecode
messages: + msg381501
2020-03-27 02:48:40rhettingersetstatus: open -> closed

nosy: + rhettinger
messages: + msg365130

resolution: duplicate
stage: resolved
2020-03-27 00:49:46pablogsalsetnosy: + pablogsal
messages: + msg365124
2020-03-27 00:18:55BTaskayasetmessages: + msg365122
2020-03-27 00:03:43eric.smithsetnosy: + eric.smith
messages: + msg365121
2020-03-26 23:03:50BTaskayasetmessages: + msg365119
2020-03-26 23:03:00BTaskayacreate