This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ghost43
Recipients ghost43
Date 2022-03-25.19:47:09
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1648237630.14.0.211482051392.issue47123@roundup.psfhosted.org>
In-reply-to
Content
Currently `ZipFile.writestr` writes the local time into the ZipFile.
(depends on both current time and local timezone)
See https://github.com/python/cpython/blob/20e6e5636a06fe5e1472062918d0a302d82a71c3/Lib/zipfile.py#L1816-L1817

This makes pip installing a package generate non-reproducible build artifacts.

Specifically, `Scripts/*.exe` files (created for packages that define entry_points/console_scripts) are not reproducible on Windows when installed by pip. This also leaks into the `*.dist-info/RECORD` files.

For example, after running `pip install wheel` or `pip install pyinstaller`,
in `wheel-0.37.1.dist-info/RECORD`, I have this line:
```
../../Scripts/wheel.exe,sha256=u9TbPw2XNs_F9uy7y2zwumuzAZDbOSB7BXjLHZ0tTHg,97103
```  
in `pyinstaller-4.10.dist-info/RECORD`, I have these lines:
```
../../Scripts/pyi-archive_viewer.exe,sha256=nC-9muPlIhUC1qvFkXHpyKJyRQqXISXxbUPXQ1XVOiM,97133
../../Scripts/pyi-bindepend.exe,sha256=udFHiAdndPpSwaIqmhmLEy36IUs1cNNoNQznSEnLJQQ,97128
../../Scripts/pyi-grab_version.exe,sha256=3ET9E841tFWujFL99aG4frzgwlP9f9pAkMgE0k2UGK0,97131
../../Scripts/pyi-makespec.exe,sha256=dJkfmITdLJhyPngmqziqqj5tH9qqfeQc5BTubeoXWUs,97127
../../Scripts/pyi-set_version.exe,sha256=sWmcOVS93fUY-wbdoz6ixBCvjy1tC4Aaw30DMmrmo-0,97130
../../Scripts/pyinstaller.exe,sha256=haInbhH0pImJn24cW4v917oUZmzXZj8OE89KFh4MO2Y,97112
```

Upon comparing multiple `Scripts/wheel.exe` files, I've found that the only difference is due to the above-mentioned timestamp embedded inside the exe (or rather, same timestamp embedded twice).

The `exe` files get created by `distlib` (vendored by pip).
Here is a traceback with an artificial exception to illustrate the codepath:
```
(env) PS C:\tmp> pip install --no-build-isolation pyinstaller
Collecting pyinstaller
  Using cached pyinstaller-4.10-py3-none-win_amd64.whl (2.0 MB)
Requirement already satisfied: setuptools in c:\tmp\env\lib\site-packages (from pyinstaller) (61.0.0)
Requirement already satisfied: pyinstaller-hooks-contrib>=2020.6 in c:\tmp\env\lib\site-packages (from pyinstaller) (2022.3)
Requirement already satisfied: altgraph in c:\tmp\env\lib\site-packages (from pyinstaller) (0.17.2)
Requirement already satisfied: pefile>=2017.8.1 in c:\tmp\env\lib\site-packages (from pyinstaller) (2021.9.3)
Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\tmp\env\lib\site-packages (from pyinstaller) (0.2.0)
Requirement already satisfied: future in c:\tmp\env\lib\site-packages (from pefile>=2017.8.1->pyinstaller) (0.18.2)
Installing collected packages: pyinstaller
ERROR: Exception:
Traceback (most recent call last):
  File "C:\tmp\env\lib\site-packages\pip\_internal\cli\base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "C:\tmp\env\lib\site-packages\pip\_internal\cli\req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "C:\tmp\env\lib\site-packages\pip\_internal\commands\install.py", line 405, in run
    installed = install_given_reqs(
  File "C:\tmp\env\lib\site-packages\pip\_internal\req\__init__.py", line 73, in install_given_reqs
    requirement.install(
  File "C:\tmp\env\lib\site-packages\pip\_internal\req\req_install.py", line 769, in install
    install_wheel(
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 729, in install_wheel
    _install_wheel(
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 646, in _install_wheel
    generated_console_scripts = maker.make_multiple(scripts_to_generate)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 440, in make_multiple
    filenames.extend(self.make(specification, options))
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 427, in make
    return super().make(specification, options)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 429, in make
    self._make_script(entry, filenames, options=options)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 329, in _make_script
    self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 263, in _write_script
    raise Exception(f"heyheyhey2. {sha256(launcher)=}. {sha256(shebang)=}. {sha256(zip_data)=}. " +
Exception: heyheyhey2. sha256(launcher)='a00a877acefc'. sha256(shebang)='58628e924f22'. sha256(zip_data)='a423496a0482'. ('SOURCE_DATE_EPOCH' in os.environ)=True
```
The interesting code is here:
https://github.com/pypa/distlib/blob/d0e3f49df5d1aeb9daeaaabf0391c9e13e4a6562/distlib/scripts.py#L251-L252
This calls into the cpython standard library, where `time.time()` gets written into the file:
https://github.com/python/cpython/blob/20e6e5636a06fe5e1472062918d0a302d82a71c3/Lib/zipfile.py#L1816-L1817

Ideally, either `distlib` or the stdlib  `zipfile` module should be changed to respect `SOURCE_DATE_EPOCH`, but it's not entirely clear to me which...

The attached patch changes `ZipFile.writestr` to respect `SOURCE_DATE_EPOCH` if set.


related https://github.com/pypa/distlib/issues/164
related https://github.com/spesmilo/electrum/issues/7739
History
Date User Action Args
2022-03-25 19:47:10ghost43setrecipients: + ghost43
2022-03-25 19:47:10ghost43setmessageid: <1648237630.14.0.211482051392.issue47123@roundup.psfhosted.org>
2022-03-25 19:47:10ghost43linkissue47123 messages
2022-03-25 19:47:09ghost43create