This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Support pointing frozen modules to the corresponding source files, if available.
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, lemburg, ncoghlan
Priority: low Keywords:

Created on 2015-09-29 12:45 by eric.snow, last changed 2022-04-11 14:58 by admin.

Messages (3)
msg251851 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-09-29 12:45
(a generalization of issue #21335)

One way or another, we should be able to record the appropriate path in the resulting data when a module is frozen.  Keeping track of the source location is useful when debugging, especially with tracebacks.  Also see a related conversation on python-dev. [1]

If the original filename for a frozen module is preserved, several possibilities open up:

* the `get_source` method of `importlib._bootstrap.FrozenImporter` could return the original source
* the traceback module could work around the co_filename issue pointed out in #21335
* the filename could be incorporated into the value set for co_filename and __file__
* frozen modules could be optionally reloaded from source

Given that the filename would (likely) be available only through loading the module, it *might* not make sense to store it on the module's spec.  However, it might also make sense to store the filename in a way that it can be exposed during the "find" stage of import.  In that case it would arguably *belong* on the module's spec.

Note that there's a chance that the original filename for a frozen module is no longer available.  That shouldn't be much of a problem, though.


[1] https://mail.python.org/pipermail/python-dev/2014-April/134097.html
msg251867 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2015-09-29 16:02
I'm not sure which kind of use for frozen modules you have in mind.

The freeze tool allows you to replace paths in source path prefixes using the -r option. We use this in eGenix PyRun to set the prefix to "<pyrun>" since at run-time, the source files won't be available anyway and we want the users to see "this file is built into pyrun".

A traceback looks like this:
      ...
      File "<pyrun>/configparser.py", line 773, in get
      File "<pyrun>/configparser.py", line 374, in before_get
      File "<pyrun>/configparser.py", line 423, in _interpolate_some
    configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%12345'

which is quite readable, IMO.

Without -r, the original file path is added, so you must be looking at some other use case.
msg277322 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-09-24 17:15
I agree with Marc-Andrew that it's hard to do anything more useful here than "-r" already does, as we can assume the source code won't be available on the target machine - it's the equivalent of having C/C++ debugging symbols available for C/C++ traceback generation.

It may still be worth doing specifically for the sake of _frozen_importlib (as that usually *does* have the importlib._bootstrap code available at runtime), but I think it would be pretty specific to that particular case.

A more generally applicable feature would be a utility to take a traceback of the form created by -r (as shown by Marc-Andre above), and converting that back to a full traceback given a directory structure that matched the layout of the frozen modules.
History
Date User Action Args
2022-04-11 14:58:21adminsetgithub: 69455
2020-06-05 18:31:23brett.cannonsetnosy: - brett.cannon
2016-09-24 17:15:58ncoghlansetmessages: + msg277322
2015-09-29 16:02:42lemburgsetmessages: + msg251867
2015-09-29 12:48:04eric.snowsetnosy: + lemburg
2015-09-29 12:45:32eric.snowcreate