This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Provide pure-Python implementation of Programs/_freeze_module for cross building
Type: enhancement Stage: resolved
Components: Build, Cross-Build Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: Alex.Willmer, christian.heimes, eric.snow, gvanrossum
Priority: normal Keywords: patch

Created on 2021-12-01 15:34 by christian.heimes, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29899 merged christian.heimes, 2021-12-02 22:23
Messages (8)
msg407459 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-12-01 15:34
As of today 3.11-dev requires Programs/_freeze_module to compile Python code to byte code. The dependency on an extra and special tool makes cross building more complicated. The _freeze_module is trivial and can be easily implemented in pure Python code. The re-introduction of _bootstrap_python helper would allow us to use a pure Python freeze_module for all modules except two importlib bootstrap modules. I have a proof of concept implementation that works for me.

Proposal for standard builds:
- Use Programs/_freeze_module to create Python/frozen_modules/importlib._bootstrap.h and Python/frozen_modules/importlib._bootstrap_external.h
- build _bootstrap_python
- use new tool "./_bootstrap_python Tools/scripts/freezemodule.py" to create remaining Python/frozen_modules/*.h

Proposal for cross builds:
- use user-provider Python interpreter (./configure --with-build-python) to generate all Python/frozen_modules/*.h with new freezemodule.py tool.

When cross building the user must already provide a Python interpreter with same version and bytecode version as the build target. Users would no longer need to provide a _freeze_module binary with ./configure --with-freeze-module.
msg407465 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2021-12-01 16:08
IIRC, Programs/_freeze_module.c was originally a C implementation of the basic functionality in Tools/freeze/freeze.py.
msg407466 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2021-12-01 16:08
Ideally we would not have two scripts that do the same thing.
msg408134 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-12-09 15:45
Eric, could you review my PR, please? It simplifies cross building to other platforms.

I took a look at Tools/freeze/freeze.py. The freeze tool creates different output than Program/_freeze_module.c.
msg408315 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-12-11 16:29
In his code review Eric made a point that the relationship of variables and their impact on normal and cross builds are not obvious. I'm going to introduce new variables for freezing and freezing dependencies. Bonus: Cross builds no longer build non-functional _bootstrap_python and Programs/_freeze_module.

Normal build:

PYTHON_FOR_FREEZE=./_bootstrap_python
FREEZE_MODULE_BOOTSTRAP=./Programs/_freeze_module
FREEZE_MODULE_BOOTSTRAP_DEPS=Programs/_freeze_module
FREEZE_MODULE=$(PYTHON_FOR_FREEZE) $(srcdir)/Programs/_freeze_module.py
FREEZE_MODULE_DEPS=_bootstrap_python $(srcdir)/Programs/_freeze_module.py


Cross build:

PYTHON_FOR_FREEZE=/path/to/build/python
FREEZE_MODULE_BOOTSTRAP=$(PYTHON_FOR_FREEZE) $(srcdir)/Programs/_freeze_module.py
FREEZE_MODULE_BOOTSTRAP_DEPS=$(srcdir)/Programs/_freeze_module.py
FREEZE_MODULE=$(FREEZE_MODULE_BOOTSTRAP)
FREEZE_MODULE_DEPS=$(FREEZE_MODULE_BOOTSTRAP_DEPS)
msg408477 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-12-13 19:48
New changeset eb483c46d62707bdf705491f76cf1fa9642fb47e by Christian Heimes in branch 'main':
bpo-45949: Pure Python freeze module for cross builds (GH-29899)
https://github.com/python/cpython/commit/eb483c46d62707bdf705491f76cf1fa9642fb47e
msg408478 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-12-13 19:53
Thanks for your review, Eric!
msg408481 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2021-12-13 20:32
You're welcome!  Thanks for doing the work!
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90107
2021-12-13 20:32:52eric.snowsetmessages: + msg408481
2021-12-13 19:53:06christian.heimessetstatus: open -> closed
resolution: fixed
messages: + msg408478

stage: patch review -> resolved
2021-12-13 19:48:54christian.heimessetmessages: + msg408477
2021-12-11 16:29:01christian.heimessetmessages: + msg408315
2021-12-09 15:45:22christian.heimessetmessages: + msg408134
2021-12-02 22:23:30christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request28123
2021-12-01 16:08:58eric.snowsetmessages: + msg407466
2021-12-01 16:08:29eric.snowsetnosy: + eric.snow
messages: + msg407465
2021-12-01 15:34:28christian.heimescreate