classification
Title: Expose PyImport_FrozenModules in imp
Type: Stage:
Components: Interpreter Core Versions: Python 2.3
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: Nosy List: jvr, loewis, theller
Priority: normal Keywords: patch

Created on 2002-11-22 22:56 by jvr, last changed 2002-11-29 20:52 by jvr. This issue is now closed.

Files
File name Uploaded Description Edit
import.c.patch jvr, 2002-11-23 23:26 expose the builtin frozen modules list
Messages (11)
msg41739 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-22 22:56
The attached patch exposes the PyImport_FrozenModules
global variable. It can be written through the new
imp.set_frozenmodules() function. No read functionality
has been provided since that would be harder to do
<wink>, and would IMO not be all that useful.

Rationale:
It allows py2exe/McMillan-Installer-like tools to be
written WITHOUT __import__ hooks, while still being
able to bootstrap with pure Python code.

Use case:
I'm in the process of writing a freeze-like tool for
MacOSX. Applications on this platform are (usually)
simply a directory containing some meta files and
arbitrary stuff. I can (and will) make it simply add
.pyc files to this directory, but it will be a huge win
to be able to put a single file in there containing all
needed Python modules. I would prefer to do this
without resorting to __import__ hooks, as they add
overhead and brittleness. I use a vanilla python
executable and bootstrap in Python. Look ma, no C!

Implementation:
PyImport_FrozenModules is an array of structs
containing a pointer to a C string (the name), a
pointer to the marshalled code object data and an int
specifying the size of the code data (negated if the
module is a package). None of these are Python objects.
imp.set_frozenmodules() takes a sequence of tuples as
its argument: (name, codedata, ispkg). Internally this
sequence is converted to a tuple, and a reference is
kept in an additional static variable in import.c. The
elements of the newly allocated PyImport_FrozenModules
array simply point to the actual string data, as
referenced by the tuple.

The patch doesn't include documentation yet, but I will
add that as soon as the patch is otherwise accepted.
msg41740 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-22 23:21
Logged In: YES 
user_id=92689

(doop: the first patch I uploaded contained a syntax error.
How embarrassing... The current file is correct.)
msg41741 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-23 00:35
Logged In: YES 
user_id=92689

Ok, I changed my mind and created a matching
imp.get_frozenmodules() function that _returns_ a Python
representation of PyImport_FrozenModules. Replaced the patch.
msg41742 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-11-23 22:14
Logged In: YES 
user_id=21627

Can you provide a patch for libimp.tex as well?
msg41743 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-23 23:26
Logged In: YES 
user_id=92689

Yes, done in the replaced patch.

I've also changed a subtlety in get_frozenmodules(): the
previous version returned the original tuple as it was set
by set_frozenmodules(), now it always builds the tuple from
scratch: it's possible that someone changes
PyImport_FrozenModules from C, invalidating the cached tuple.
msg41744 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2002-11-25 08:52
Logged In: YES 
user_id=11105

I haven't actually tried out the patch, but it looks fine to me.
msg41745 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2002-11-25 19:22
Logged In: YES 
user_id=11105

Hm, shouldn't overwriting the frozen module table, if it is 
already set, raise an error?

Or should there be more than one frozen tables?

And having said that, I could even think of situations where 
the frozen module tables should be private to *one* 
interpreter, and not be shared if there have been other ones 
created by Py_NewInterpreter. But this issue should probably 
left for a separate patch ;-)
msg41746 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-25 19:29
Logged In: YES 
user_id=92689

[raise error on overwrite]
No, for one because it is always set (see Python/frozen.c or
do "import __hello__" ;-). Also: it _might_ be useful to
extend the list by first retrieving is, appending to it and
setting it again.

The other "issue": let's make that an issue when rewriting
the entire import stuff is an active project... (It would
also involve a C API to set/modify the frozen modules list,
which, erm, perhaps should be a dict anyway, and and...)
msg41747 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-26 00:53
Logged In: YES 
user_id=92689

Anticipating acceptance of this patch, I've checkin in
actual (but optional) employment of imp.set_frozenmodules()
in Mac/Lib/bundlebuilder.py.

The neat thing about this is that bootstrapping happens in a
custom "site" module: I don't even have to wrap the main
program...
msg41748 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-29 12:33
Logged In: YES 
user_id=92689

I'm ready to commit this. Any objections?
msg41749 - (view) Author: Just van Rossum (jvr) * Date: 2002-11-29 20:52
Logged In: YES 
user_id=92689

Checked in with minor improvements as:
Misc/NEWS rev. 1.550
Doc/lib/libimp.tex rev. 1.33
Python/import.c rev. 2.211
History
Date User Action Args
2002-11-22 22:56:04jvrcreate