classification
Title: Deprecate imp.find_module()/load_module()
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: 17314 18055 18157 18158 Superseder:
Assigned To: brett.cannon Nosy List: Arfrever, brett.cannon, eric.araujo, eric.snow, ezio.melotti, flox, terry.reedy, theller
Priority: low Keywords:

Created on 2012-05-13 16:32 by brett.cannon, last changed 2013-06-11 21:13 by brett.cannon. This issue is now closed.

Messages (8)
msg160522 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-05-13 16:32
With importlib.find_loader() now implemented, imp.find_module() (and its companion, imp.load_module()) can go away and stop inflicting their bad API upon the world.

The trick, though, is fixing code that uses the API. That means pkgutil, multiprocessing.forking, modulefinder, and idlelib.EditorWindow all need to be patched to stop using imp.find_module()/load_module().
msg160525 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-05-13 17:07
http://hg.python.org/lookup/59870239813c documents imp.find_module/load_module as deprecated w/o actually raising the deprecation. The code works fine, but the API is just crap. So in the name of ease of transition (and my own personal sanity), I didn't go full-blown deprecation. The remaining code that uses imp.find_module() in the stdlib uses the API's unique properties (e.g. executing under a different name in multiprocessing) or directly exposes the API (e.g. pkgutil and modulefinder). As for idle, I don't run the tool so I don't feel like editing the code and getting it wrong.
msg160526 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-05-13 17:07
BTW, I'm leaving this issue open in case someone wants to take a stab and removing the uses of imp.find_module()/load_module() from the stdlb.
msg183180 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-27 20:04
Assuming issue #17314 gets fixed, that leaves the following uses of imp.find_module():

Lib/idlelib/EditorWindow.py
39:    """Version of imp.find_module() that handles hierarchical module names"""
45:        (file, filename, descr) = imp.find_module(tgt, path)

Lib/modulefinder.py
482:        return imp.find_module(name, path)

Lib/pkgutil.py
229:            file, filename, etc = imp.find_module(subname, path)

Lib/test/test_imp.py
59:            with imp.find_module('module_' + mod, self.test_path)[0] as fd:
64:            imp.find_module('badsyntax_pep3120', path)
68:            fp, filename, info  = imp.find_module('module_' + mod,
77:        fp, filename, info = imp.find_module("tokenize")
91:            file, filename, info = imp.find_module(temp_mod_name)
147:            file, filename, info = imp.find_module(temp_mod_name)
187:                          imp.find_module, "badsyntax_pep3120", [path])
201:            x = imp.find_module("os")
213:        x = imp.find_module(example)
223:        fileobj, pathname, description = imp.find_module(m)

Lib/test/test_import.py
128:                imp.find_module, TESTFN, ["."])

Lib/test/test_importhooks.py
118:            file, filename, stuff = imp.find_module(subname, path)
msg183181 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-27 20:05
The remaining uses of imp.load_module() after issue #17314 (which will probably all disappear as uses of imp.find_module() is removed):

Lib/idlelib/EditorWindow.py
48:        module = imp.load_module(tgt, file, filename, descr)

Lib/pkgutil.py
291:            mod = imp.load_module(fullname, self.file, self.filename, self.etc)

Lib/pydoc.py
47:#   - imp.load_module() cannot be prevented from clobbering existing
277:            module = imp.load_module(name, file, path, (ext, 'r', kind))

Lib/test/test_imp.py
155:                mod = imp.load_module(temp_mod_name, file, filename, info)
203:            new_os = imp.load_module("os", *x)
217:        mod = imp.load_module(example, *x)

Lib/test/test_importhooks.py
132:        mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
msg190763 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-06-07 17:29
Current status (with test_imp stripped out):

> ack "imp\.(find|load)_module" Lib
Lib/modulefinder.py
482:        return imp.find_module(name, path)

Lib/pkgutil.py
189:            file, filename, etc = imp.find_module(subname, path)
251:            mod = imp.load_module(fullname, self.file, self.filename, self.etc)

Lib/pydoc.py
47:#   - imp.load_module() cannot be prevented from clobbering existing
280:            module = imp.load_module(name, file, path, (ext, 'r', kind))

Lib/test/test_importhooks.py
118:            file, filename, stuff = imp.find_module(subname, path)
132:        mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
msg190764 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-06-07 17:40
The modulefinder usage is directly exposed in its API as the return value of a find_module method, which makes removal a pain. Adding Thomas Heller to see what he has to say.

The pkgutil usage is in classes that have been deprecated, so don't care.

The rest should be workable and I will file separate bugs.
msg190813 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2013-06-08 18:02
> The modulefinder usage is directly exposed in its API as the return
> value of a find_module method, which makes removal a pain. Adding
> Thomas Heller to see what he has to say.

Some months ago, I have started to implement a brand-new modulefinder,
which is based on importlib.  It has a different API, but the (barely)
documented API functions remain the same or could be made compatible.
The advantage against the current modulefinder is that it finds modules
in zipfiles, for example in zipped eggs.

IMO it is not yet ready for inclusion into Python 3.4, but given the
plans for the first alpha release in August I will hopefully find
time to work on it a little bit more.

If someone wants to read the code, or try it out:

Modulefinder:
http://code.google.com/p/ctypes-stuff/source/browse/trunk/mf/mf4.py

Test:
http://code.google.com/p/ctypes-stuff/source/browse/trunk/mf/maketest.py

Thomas
History
Date User Action Args
2013-06-11 21:13:25brett.cannonsetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2013-06-08 18:02:04thellersetmessages: + msg190813
2013-06-07 17:44:14brett.cannonsetdependencies: + remove usage of imp.load_module() from pydoc
2013-06-07 17:43:36brett.cannonsetdependencies: + Delete test_importhooks
2013-06-07 17:40:55brett.cannonsetnosy: + theller
messages: + msg190764
2013-06-07 17:29:00brett.cannonsetmessages: + msg190763
versions: + Python 3.4, - Python 3.3
2013-05-25 15:40:26brett.cannonsetdependencies: + Stop using imp in IDLE
2013-02-27 20:06:35brett.cannonsetassignee: brett.cannon
2013-02-27 20:05:34brett.cannonsetmessages: + msg183181
2013-02-27 20:04:26brett.cannonsetmessages: + msg183180
2013-02-27 20:03:22brett.cannonsetdependencies: + Stop using imp.find_module() in multiprocessing
2013-02-27 20:03:15brett.cannonlinkissue17177 dependencies
2012-07-11 16:22:17floxsetnosy: + flox
2012-07-11 04:18:42eric.araujosetnosy: + eric.araujo
2012-07-09 20:10:51brett.cannonunlinkissue13959 dependencies
2012-05-18 17:54:53terry.reedysetnosy: + terry.reedy
2012-05-13 19:47:48eric.snowsetnosy: + eric.snow
2012-05-13 18:12:32Arfreversetnosy: + Arfrever
2012-05-13 17:07:54brett.cannonsetmessages: + msg160526
2012-05-13 17:07:21brett.cannonsetpriority: normal -> low

messages: + msg160525
2012-05-13 16:35:07ezio.melottisetnosy: + ezio.melotti
2012-05-13 16:32:23brett.cannonlinkissue13959 dependencies
2012-05-13 16:32:08brett.cannoncreate