classification
Title: Implementation for PEP 451 (importlib.machinery.ModuleSpec)
Type: enhancement Stage: patch review
Components: Interpreter Core, Library (Lib) Versions: Python 3.4
process
Status: open Resolution:
Dependencies: 19697 19700 19701 19703 19704 19705 19706 19707 19708 19709 19710 19711 19712 19713 19714 21098 21099 Superseder:
Assigned To: eric.snow Nosy List: Arfrever, Claudiu.Popa, berker.peksag, brett.cannon, eric.snow, larry, ncoghlan, pconnell, python-dev, terry.reedy
Priority: deferred blocker Keywords: patch

Created on 2013-08-28 06:56 by eric.snow, last changed 2014-03-29 21:42 by eric.snow.

Files
File name Uploaded Description Edit
modulespec-initial.diff eric.snow, 2013-10-05 05:07 review
import-system-reference.diff eric.snow, 2013-10-26 21:43 review
modulespec-primary-changes.diff eric.snow, 2013-11-01 22:25 review
issue18864-has-location-setter.diff eric.snow, 2013-12-10 03:29 review
Messages (61)
msg196352 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-08-28 06:56
This ticket will track the implementation for PEP 451 (ModuleSpec).  I'll have a patch up in the next couple days.
msg198981 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-10-05 05:07
Here's an initial patch.  Some key things left to do:

* unit tests
* documentation
* implement exec_module() for the various importlib loaders.

Once that's squared away there are further things that will be addressed in new tickets (or at least separate patches).  This includes:

* remove init_module_attrs() and module_to_load()
* deprecations
* clear a bunch of helper functions out of _bootstrap.py
* use ModuleSpec with __main__
* address impact on stdlib (pkgutil, pickle, etc.)
msg201210 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-10-25 06:40
Here's a patch just for the changes to the import page in the language reference.  It's not perfect, but should represent the target pretty closely.
msg201212 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-10-25 06:47
Let's try that again with the proper diff. :)
msg201224 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-10-25 08:01
Thanks, that looks pretty good - so much less work for loaders to do.

After this, we may even be able to finally fix the circular import bug and the one where submodule imports succeed but the overall package import fails.
msg201244 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-10-25 15:37
I've added a review
msg201369 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-10-26 15:51
Something else we may be able to fix (albeit probably not in 3.4): several of the issues noted in PEP 395 (since __main__.__spec__.name will give the real module name, even when __name__ is set to "__main__" or otherwise modified)
msg201399 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-10-26 21:43
Here's an update of the language reference patch based on Brett's comments.
msg201604 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-10-29 05:34
Here's an updated patch that implements the meat of the current PEP.  Docs are still lacking and I could probably add a few more tests.  Other key things left to do:

* deprecations and removals
* refactor importlib finders and loaders to use the new Finder/Loader APIs
* refactor pythonrun.c to make use of __spec__
* adjust other APIs to use __spec__ (pickle, etc.)
msg201605 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-10-29 05:40
This time against tip. :)

I should also point out that this patch is on top of the path in issue19413.
msg201946 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-01 22:21
Here's a new patch that is mostly up to date with the PEP.  I still need to work on Docs and add more tests.  There are a few failing tests (due to the recent reload patch) that I need to fix when I get a minute.

This patch also implements find_spec() on FileFinder and PathFinder.  Left to do:

* deprecations and removals
* refactor importlib loaders to use the new Finder/Loader APIs
* refactor pythonrun.c to make use of __spec__
* adjust other APIs to use __spec__ (pickle, etc.)
msg202456 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-09 03:53
I've created a server-side clone for the implementation:

http://hg.python.org/features/pep-451/
msg202461 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-09 05:18
Okay, I've updated the pep451 branch in the clone to include as much of the implementation as I've completed, which is the bulk of the functional changes.  It's enough to pass the test suite.

Here's what I'd like to get done before the feature freeze (in this priority order):

1. wrap up the current functional changes in the clone;
2. change module.__initializing__ to module.__spec__._initializing;
3. refactor importlib loaders to use the new Finder/Loader APIs;
4. refactor pythonrun.c to make use of __spec__;
5. implement the deprecations and removals;
6. adjust other APIs to use __spec__ (pickle, etc.);
7. add rudimentary doc additions for the new APIs.

Other things that can (but don't have to) wait until after the beta release:

* finish doc changes;
* fill in gaps in test coverage (there shouldn't be much due to Brett's mother of all test suites for importlib)

I haven't had a chance yet to make any changes to Doc/reference/import.rst in response to Brett's review, but I did make the Doc/library/importlib.rst changes he recommended.
msg202482 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-09 15:58
I don't quite know what you mean by "current functional changes in the clone". Is this changes to import to use exec_module() and such? Just a guess since step 3 suggests there are no loaders implementing the new API currently.

As for the tests, I'm hoping that simply refactoring some of them will allow reusing a bunch of the finder & loader tests instead of having to do it from scratch.
msg202483 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-09 16:00
I should clarify why I want a clarification on step 1. On Friday my current plan (barring other bugs I need to squash for b1) is to just start working on the pep-451 repo and I want a TODO list to follow in this issue so I don't have to waste time trying to figure out what to work on next. That's why I want it spelled out if we want the loaders fixed first or import, etc.
msg202655 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-11 22:40
Sorry that wasn't more clear.  I committed the changes from the modulespec-primary-changes.diff patch to the pep451 branch in the server-side clone.  Those changes are:

Step 1
------
1. added ModuleSpec class
2. added _SpecMethods wrapper class
3. added ModuleSpec factory functions
4. added tests for ModuleSpec, _SpecMethods, and the factories
5. exposed ModuleSpec in importlib.machinery
6. exposed ModuleSpec factories in importlib.util
7. added basic docs for ModuleSpec and the factory functions

Step 2
------
1. changed _find_module() to _find_spec()
2. changed _find_and_load_unlocked() to use _find_spec() and _SpecMethods
3. changed _setup() to use specs
4. changed pydoc to recognize __spec__

Step 3
------
1. updated the import reference doc
2. changed importlib.reload() to use specs
3. added importlib.find_spec()
4. changed importlib.find_loader() to wrap find_spec()
5. updated importlib.abc to reflect the new APIs
6. changed pkgutil to use specs
7. changed imp to use specs
8. fixed a bunch of broken tests to use spec

Step 3
------
1. implemented find_spec() on PathFinder
2. implemented find_spec() on FileFinder
3. re-implemented FileFinder.find_loader() to wrap find_spec()
4. re-implemented PathFinder.find_module() to wrap find_spec()
5. changed _NamespacePath to use specs

Step 5
------
1. added _module_repr function
2. changed ModuleType.__repr__ to wrap _module_repr

Others
------
* removed _NamespaceLoader
* added comments indicating deprecations and removals

At this point, the test suite passes and the fundamental changes of the PEP are implemented (on the server-side clone).  Here's what's left to do before the feature freeze:

1. 
2. change module.__initializing__ to module.__spec__._initializing
3. refactor importlib loaders to use the new Finder/Loader APIs
4. refactor pythonrun.c to make use of specs
5. check pkgutil for any missed changes
6. implement the deprecations and removals
7. adjust other APIs to use __spec__ (pickle, runpy, inspect, pydoc, others?)
8. evaluate any impact on setuptools

Other things that can (but don't have to) wait until after the beta release:

* finish doc changes
* fill in any gaps in test coverage
* ensure new docstrings exist and are correct
* ensure existing docstrings are still correct
msg202656 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-11 22:41
"1." was intentionally left blank <wink>
msg202657 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-11 22:44
Regarding tests, a bunch of importlib (and other?) tests make direct calls to find_module(), find_loader(), or load_module().  These are still working, as expected.  However, we should probably either replace them or supplement them with equivalent tests that make use of specs directly.
msg202658 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-11 23:19
Yeah, don't replace any tests, add new ones for the new APIs. Given the
needs of 2/3 compatible loader implementations, the deprecations referred
to in the PEP should also be documentation-only for 3.4.

A more conservative approach also gives us a chance to make sure we have
provided a full replacement for load_module - it occurred to me after the
PEP was accepted that we may eventually need a "can_load_into(target)"
loader API after all, since loaders may not be finder specific. That means
that with the current API of passing the target to find_spec, I potentially
talked us into repeating the "load_module" mistake on the finder side of
things by asking the finder to handle more than it needed to.
msg202659 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-11 23:36
> Yeah, don't replace any tests, add new ones for the new APIs.

That's what makes the most sense to me too.

> Given the
> needs of 2/3 compatible loader implementations, the deprecations referred
> to in the PEP should also be documentation-only for 3.4.

I'm fine with that.  I seem to remember one place where an actual deprecation warning would be good, but I'll have to double-check.

> A more conservative approach also gives us a chance to make sure we have
> provided a full replacement for load_module - it occurred to me after the
> PEP was accepted that we may eventually need a "can_load_into(target)"
> loader API after all, since loaders may not be finder specific. That means
> that with the current API of passing the target to find_spec, I potentially
> talked us into repeating the "load_module" mistake on the finder side of
> things by asking the finder to handle more than it needed to.

Nice analogy.  "can_load_into()" makes sense.  Do you think it's important enough to include in 3.4?  I'm not sure it is, since I'd expect it to be a pretty uncommon case that can be worked around pretty easily (since the finder is fully aware of the loader it's creating).  The extra loader method would help with that boilerplate operation, but...

As we found out (and you expounded) there are a variety of reload/load-into cases that we could address more explicitly.  Perhaps there's a better API that could address those needs more broadly, or maybe they're just not worth addressing specifically.  Exploring all this is something that can wait, IMHO.
msg202660 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-11 23:41
Although, a boolean query method would bring back the problem of the loader
not reporting any details on *why* it can't load into a particular target
module. So it may be better to have an optional loader
"check_existing_target" API that throws a suitable exception if the target
is unacceptable. Loaders that don't support reloading at all would just
always raise an exception, while those that don't care would just not
implement the method.

That would address my concern about the lack of useful error information in
Eric's original boolean check idea, without bothering finders with loader
related details as the accepted PEP does (I still like the idea of passing
a target to importlib.find_spec - I just no longer think we should be
passing that down to the finders themselves).

Another thing we need to check we have a test for: ensuring reloading a
namespace module picks up new directories added since it was first loaded.

This would all be so much easier if reloading wasn't supported in the first
place :)
msg202661 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-11 23:49
On 12 Nov 2013 09:36, "Eric Snow" <report@bugs.python.org> wrote:
>
> As we found out (and you expounded) there are a variety of
reload/load-into cases that we could address more explicitly.  Perhaps
there's a better API that could address those needs more broadly, or maybe
they're just not worth addressing specifically.  Exploring all this is
something that can wait, IMHO.

Yes, that's an option, too, and probably a good one. To go down that path,
we would drop the various "target" parameters and say loaders that need to
check for the reloading case should continue to provide load_module without
exec_module (at least for 3.4). runpy would use the rule that it supports
anything that exposes exec_module without create_module.
msg202662 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-11 23:55
> (I still like the idea of passing
> a target to importlib.find_spec - I just no longer think we should be
> passing that down to the finders themselves).

Passing the target to the finders isn't just for the sake of any implicit "check_target" test, though that was the original motivator.  It also allows the finder to decide between multiple loaders based on other criteria related to the target (but not necessarily the loader).  I think it was a good addition to the API regardless.

> Another thing we need to check we have a test for: ensuring reloading a
> namespace module picks up new directories added since it was first loaded.

Agreed.  Furthermore, such a test is worthwhile outside the context of PEP 451.  I'm tempted to say we're already covered with existing tests, but reload is goofy enough that an explicit test is worth it.

> This would all be so much easier if reloading wasn't supported in the first
> place :)

So very true. :)
msg202663 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-11 23:59
And coming full circle: there's no *harm* in letting finders reject loading
into a target module, and that's orthogonal to having loaders reject it.
It's just that loaders that want to do that will currently still need to
implement load_module.

That means the only thing we need to postpone is the load_module
deprecation, since it still covers at least one advanced use case the new
API doesn't handle.
msg202664 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 00:06
Sounds good.  It will be worth adding a note to the load_module() docs indicating the limited cases where it is still appropriate, and encouraging the use of exec_module() instead.
msg202669 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 04:16
I'm running the pep-451 clone against 2 buildbots for now:

* http://buildbot.python.org/all/builders/x86%20Windows%20Server%202003%20%5BSB%5D%20custom
* http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%20custom

The ubuntu one is happy with the feature clone, but the windows buildbot keeps finding problems.  At first it was issue17116 all over again.  I've now set __spec__ to None by default (a la issue17115).  Then it fussed about test_everyone_has___loader__.  I'm sure something else will turn up.

Any thoughts on why import-related tests are responding differently depending on platform?  FWIW, I think the Windows failures should have also failed on the Ubuntu buildbot.

I'm also going to run on a Windows 7 and OS X to see where things stand.
msg202672 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 05:04
The OS X buildbot I used [1] did not exhibit any of the failures that Windows server 2003 showed.

[1] http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%20custom
msg202673 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 05:07
The Windows 7 buildbot I tried [1] shows the exact same failures that the server 2003 does.

[1] http://buildbot.python.org/all/builders/x86%20Windows7%20custom
msg202676 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 07:44
Naturally the other Windows buildbot I tried did not fail any import-related tests.  So much for consistency. :(

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%20custom
msg202681 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-12 10:16
The three Windows machines look like they have very different system
versions of Python (ActiveState, cygwin, vanilla CPython). Any chance we're
accidentally invoking the system Python instead of the built one somewhere?
msg202695 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-12 15:34
I think worrying about more API expansions to handle edge case/advanced reload scenarios is not worth it (or at least not in Python 3.4). And if we are shifting importlib.reload() back to looking for a new finder then we are only worrying about when people decide to directly call exec_module() manually and want some pre-condition way to verify that the reload would succeed w/o putting that logic into exec_module() itself.

All I'm seeing is a mole hill for some advanced users who should be able to deal with it on their own.
msg202710 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 18:03
> The three Windows machines look like they have very different system
> versions of Python (ActiveState, cygwin, vanilla CPython). Any chance we're
> accidentally invoking the system Python instead of the built one somewhere?

Good point.  I'll check that out.  That said, on my local clone (Ubuntu 12.10) I'm also getting no failures.  It could also be related to the speed of the buildbot, but I doubt it.  Anyway, I'll take a closer look at the stdout artifacts.
msg202711 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 18:04
> I think worrying about more API expansions to handle edge case/advanced
> reload scenarios is not worth it (or at least not in Python 3.4).

Agreed.
msg202712 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-12 18:06
Regarding the to-do list, I'm going to add a TODO file to the root of the feature clone (in the pep451 branch) so we don't have to keep updating this ticket. :)
msg202917 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-15 04:47
FYI, as of cc7ebd952777 the bulk of the implementation in features/pep-451 is working.

There is a failing test on the Windows Server 2003 buildbot (which I'm trying to track down).  I haven't been able to reproduce the failure on other buildbots, on my local Ubuntu 12.10 workstation, or on my local Windows laptop.  See http://buildbot.python.org/all/builders/x86%20Windows%20Server%202003%20%5BSB%5D%20custom/builds/30/steps/test/logs/stdio.
msg202918 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-15 04:50
Cool.  Roundup linked the revision hash to the feature clone. :)
msg203329 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-18 22:47
Just a heads up - since the two of you appear to have this well in hand,
don't wait for a review from me before committing it. I have plenty of
other things to look at before the beta deadline :)
msg203340 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-19 07:36
Brett: looks like something frozen-related broke due to 6d1656ab2c85a527c.

test_frozen in test_frozen.py is failing now because frozen modules no longer have a __cached__ attribute (which was previously set to None).  Previously it was set in PyImport_ExecCodeModuleObject() in import.c.  I'm guessing that simply setting __cached__ (and __file__?) in FrozenImporter.exec_module() should restore the previous behavior.  Doing so makes the test pass at least.
msg203411 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-19 18:28
That's not the right fix for that test failure. If exec_module() is not meant to set import-related attributes like __cached__ because import itself (through _SpecMethods.init_module_attrs()) then test_frozen needs to be updated to not expect such attributes.

Honestly test_frozen should probably either be rolled into test_importlib.frozen or deleted instead of existing separately.
msg203578 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-21 07:30
I've updated the TODO a little as well as cleaning up the XXX markers.  The relevant ones are mostly just open questions on implementation tweaks, so nothing major.  Otherwise my goal is to finish as much as possible of the non-critical items before the beta.

Brett: Any opinions on priority for the non-critical items?

Also, do you have a plan in mind for "Python can function w/ find_module(), find_loader(), and load_module() removed"?  Do you mean comment out the deprecated methods and see what happens?
msg203649 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-21 16:00
Semantic-affecting changes first, then fluffy bits in terms of priority. Basically go based on what is most critical to be in 3.4.

And yes, basically can you comment out find_module/find_loader/load_module and execute the test suite (sans import-related tests) without import-related failures.
msg203800 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-22 16:30
No longer blocking b1, now just blocking rc1.
msg203801 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-22 16:31
Your commit doesn't compile on Windows.

Category 	None
Changed by 	Eric Snow <ericsnowcurrently@gmail.com>
Changed at 	Fri 22 Nov 2013 16:17:09
Branch 	default
Revision 	07229c6104b16d0ab7cc63f3306157d3d2819fed
Comments

Implement PEP 451 (ModuleSpec).


http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/3348/steps/compile/logs/stdio
msg203810 - (view) Author: Berker Peksag (berker.peksag) * Date: 2013-11-22 17:04
There is a commented-out line in Lib/importlib/_bootstrap.py:

+#    if hasattr(loader, 'get_data'):
+    if hasattr(loader, 'get_filename'):

http://hg.python.org/cpython/rev/07229c6104b1#l6.369
msg203811 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-22 17:08
Removed the commented-out line in c8a84eed9155
msg203825 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-22 18:13
Failing buildbot:

http://buildbot.python.org/all/builders/x86%20XP-4%203.x/builds/9630/steps/test/logs/stdio

test.test_module.ModuleTests.test_module_repr_source
test.test_pkgutil.ExtendPathTests.test_iter_importers (passed when retried)
msg203831 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-22 18:42
Same failures on http://buildbot.python.org/all/builders/x86%20Gentoo%203.x/builds/5444/steps/test/logs/stdio
msg203835 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-22 19:01
The test_module failure can be triggered by running test_importlib test_module. For some reason unittest's __spec__ gets set to one for the BuiltinImporter (and nothing else).
msg203843 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-22 19:38
New changeset 98bab4a03120 by Brett Cannon in branch 'default':
Issue #18864: Don't try and use unittest as a testing module for
http://hg.python.org/cpython/rev/98bab4a03120
msg205384 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-12-06 16:23
I just realized that there is no way to make ModuleSpec.has_location return True if the spec is created by simply instantiating ModuleSpec. As it stands now you **must** go through importlib.util.spec_from_file_location(), which makes it not a helper but a **required** API to use. I think this is the only thing that cannot be stated directly through ModuleSpec instantiation or public attribute assignment.

This means either (a) the docs on ModuleSpec need to be **very** clear that you must use spec_from_file_location() to make spec.has_location be true, (b) add another keyword-only argument to ModuleSpec.__init__(), or (c) make has_location settable (e.g. be able to only set it to True/False or alternatively assigning it sets both has_location to True and origin to the assigned value).
msg205520 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-12-08 06:22
Good catch.  My preference would be for (c) simply add a setter for the property that sets the underlying private variable to True or False.
msg205575 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-12-08 16:10
(c) it is then! I would just take the time to call bool() on the arg to coalesce it into a True/False thing (which maybe makes sense in __init__() as well).
msg205576 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-12-08 16:11
Actually, ignore the __init__() part of my last comment since it's set to False directly.
msg205769 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-12-10 03:29
Here's a patch for adding a setter for ModuleSpec.has_location.
msg205822 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-12-10 14:58
LGTM
msg205881 - (view) Author: Roundup Robot (python-dev) Date: 2013-12-11 05:17
New changeset e961a166dc70 by Eric Snow in branch 'default':
Issue #18864: Add a setter for ModuleSpec.has_location.
http://hg.python.org/cpython/rev/e961a166dc70
msg207509 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-01-07 05:27
About the only thing left for this ticket is to finish up writing a few tests and round out some documentation.
msg207510 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-01-07 05:32
There are a few lingering to-do comments that will need to be addressed as part of the remaining work:

Lib/test/test_importlib/extension/test_case_sensitivity.py:# XXX find_spec tests
Lib/test/test_importlib/extension/test_finder.py:# XXX find_spec tests
Lib/test/test_importlib/source/test_file_loader.py:                mod = loader.load_module('_temp') # XXX
Lib/test/test_importlib/source/test_file_loader.py:            # XXX Change to use exec_module().
Lib/test/test_importlib/test_windows.py:    # XXX Need a test that finds the spec via the registry.
msg210927 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-02-11 11:56
Failure with 3.4.0c1:

[170/394/5] test_importlib
test test_importlib failed -- Traceback (most recent call last):
  File "C:\Programs\Python34.32\lib\unittest\case.py", line 57, in testPartExecutor
    yield
  File "C:\Programs\Python34.32\lib\unittest\case.py", line 574, in run
    testMethod()
  File "C:\Programs\Python34.32\lib\unittest\loader.py", line 32, in testFailure
    raise exception
ImportError: Failed to import test module: test.test_importlib.source.test_abc_loader
Traceback (most recent call last):
  File "C:\Programs\Python34.32\lib\unittest\loader.py", line 312, in _find_tests
    module = self._get_module_from_name(name)
  File "C:\Programs\Python34.32\lib\unittest\loader.py", line 290, in _get_module_from_name
    __import__(name)
  File "C:\Programs\Python34.32\lib\test\test_importlib\source\test_abc_loader.py", line 73, in <module>
    class PyLoaderMock(abc.PyLoader):
AttributeError: 'module' object has no attribute 'PyLoader'

On the other hand, the test passes on a fresh debug build.
msg210957 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2014-02-11 16:04
Terry has admitted in other bugs he filed that he didn't use a fresh install location, so the test failure is most likely a red herring.
msg215152 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-03-29 21:42
I've gone ahead and moved all remaining work into separate issues.  Once the 4 remaining sub-issues are resolved, this one can be closed.  I've *finally* got a couple days free so I want to close this out. :)
History
Date User Action Args
2014-03-29 21:42:40eric.snowsetmessages: + msg215152
2014-03-29 21:35:55eric.snowsetdependencies: + Address remaining PEP 451-related to-do comments., Switch applicable importlib tests to use PEP 451 API
2014-02-11 16:04:52brett.cannonsetmessages: + msg210957
2014-02-11 11:56:56terry.reedysetnosy: + terry.reedy
messages: + msg210927
2014-01-07 05:32:28eric.snowsetmessages: + msg207510
2014-01-07 05:27:47eric.snowsetdependencies: - Update zipimport for PEP 451, Update pickle to take advantage of PEP 451
messages: + msg207509
2013-12-15 02:25:41brett.cannonunlinkissue17621 dependencies
2013-12-11 17:16:18eric.snowsetdependencies: - Implement _imp.exec_builtin and exec_dynamic
2013-12-11 05:17:20python-devsetmessages: + msg205881
2013-12-10 14:58:09brett.cannonsetmessages: + msg205822
2013-12-10 03:29:54eric.snowsetfiles: + issue18864-has-location-setter.diff

messages: + msg205769
2013-12-09 00:16:09hayposetnosy: - haypo
2013-12-08 16:11:31brett.cannonsetmessages: + msg205576
2013-12-08 16:10:57brett.cannonsetmessages: + msg205575
2013-12-08 06:22:40eric.snowsetmessages: + msg205520
2013-12-06 16:23:53brett.cannonsetmessages: + msg205384
2013-11-23 08:25:11pconnellsetnosy: + pconnell
2013-11-22 19:38:16python-devsetnosy: + python-dev
messages: + msg203843
2013-11-22 19:01:23brett.cannonsetmessages: + msg203835
2013-11-22 18:42:04brett.cannonsetmessages: + msg203831
2013-11-22 18:13:25eric.snowsetmessages: + msg203825
2013-11-22 17:08:07brett.cannonsetmessages: + msg203811
2013-11-22 17:04:35berker.peksagsetmessages: + msg203810
2013-11-22 17:03:01brett.cannonsetdependencies: + Add tests for importlib.machinery.WindowsRegistryFinder
2013-11-22 16:50:16brett.cannonsetdependencies: + Deprecate various things in importlib thanks to PEP 451
2013-11-22 16:48:31brett.cannonsetdependencies: + Make sure there are exec_module tests for _LoaderBasics subclasses
2013-11-22 16:47:16brett.cannonsetdependencies: + add test for changed portions after reloading a namespace package
2013-11-22 16:45:50brett.cannonsetdependencies: + Make sure documentation for PEP 451 is finished
2013-11-22 16:43:47brett.cannonsetdependencies: + Check pythonrun.c is fully using PEP 451
2013-11-22 16:43:05brett.cannonsetdependencies: + Check pkgutil for anything missing for PEP 451
2013-11-22 16:41:39brett.cannonsetdependencies: + Check if unittest.mock needs updating for PEP 451
2013-11-22 16:40:16brett.cannonsetdependencies: + Check if inspect needs updating for PEP 451
2013-11-22 16:39:30brett.cannonsetdependencies: + Update test.test_namespace_pkgs to PEP 451
2013-11-22 16:38:37brett.cannonsetdependencies: + Update test.test_threaded_import to PEP 451
2013-11-22 16:37:23brett.cannonsetdependencies: + Update pickle to take advantage of PEP 451, Update pydoc to PEP 451
2013-11-22 16:34:15brett.cannonsetdependencies: + Update multiprocessing for PEP 451
2013-11-22 16:32:13brett.cannonsetdependencies: + Update runpy for PEP 451
2013-11-22 16:31:07hayposetnosy: + haypo
messages: + msg203801
2013-11-22 16:30:04brett.cannonsetpriority: release blocker -> deferred blocker

dependencies: + Update zipimport for PEP 451
messages: + msg203800
2013-11-22 16:28:05brett.cannonsetdependencies: + Implement _imp.exec_builtin and exec_dynamic
2013-11-22 16:26:00brett.cannonsetdependencies: + Document the possible values for __main__.__spec__
2013-11-22 09:04:14Claudiu.Popasetnosy: + Claudiu.Popa
2013-11-21 16:00:09brett.cannonsetmessages: + msg203649
2013-11-21 07:30:49eric.snowsetmessages: + msg203578
2013-11-19 18:28:24brett.cannonsetmessages: + msg203411
2013-11-19 07:36:10eric.snowsetmessages: + msg203340
2013-11-18 22:47:25ncoghlansetmessages: + msg203329
2013-11-15 11:07:12ncoghlanlinkissue19593 dependencies
2013-11-15 04:50:57eric.snowsetmessages: + msg202918
2013-11-15 04:47:47eric.snowsetmessages: + msg202917
2013-11-12 18:06:56eric.snowsetmessages: + msg202712
2013-11-12 18:04:31eric.snowsetmessages: + msg202711
2013-11-12 18:03:12eric.snowsetmessages: + msg202710
2013-11-12 15:34:53brett.cannonsetmessages: + msg202695
2013-11-12 10:16:44ncoghlansetmessages: + msg202681
2013-11-12 07:44:29eric.snowsetmessages: + msg202676
2013-11-12 05:07:26eric.snowsetmessages: + msg202673
2013-11-12 05:04:16eric.snowsetmessages: + msg202672
2013-11-12 04:16:08eric.snowsetmessages: + msg202669
2013-11-12 00:06:20eric.snowsetmessages: + msg202664
2013-11-11 23:59:59ncoghlansetmessages: + msg202663
2013-11-11 23:55:36eric.snowsetmessages: + msg202662
2013-11-11 23:49:41ncoghlansetmessages: + msg202661
2013-11-11 23:41:06ncoghlansetmessages: + msg202660
2013-11-11 23:36:51eric.snowsetmessages: + msg202659
2013-11-11 23:19:27ncoghlansetmessages: + msg202658
2013-11-11 22:44:45eric.snowsetmessages: + msg202657
2013-11-11 22:41:19eric.snowsetmessages: + msg202656
2013-11-11 22:40:32eric.snowsetmessages: + msg202655
2013-11-09 16:00:17brett.cannonsetmessages: + msg202483
2013-11-09 15:58:44brett.cannonsetmessages: + msg202482
2013-11-09 05:18:46eric.snowsetpriority: normal -> release blocker
nosy: + larry
messages: + msg202461

2013-11-09 03:53:53eric.snowsetmessages: + msg202456
2013-11-01 22:26:18eric.snowsetfiles: - modulespec-primary-changes.diff
2013-11-01 22:26:10eric.snowsetfiles: - modulespec-primary-changes.diff
2013-11-01 22:25:55eric.snowsetfiles: + modulespec-primary-changes.diff
2013-11-01 22:21:21eric.snowsetfiles: + modulespec-primary-changes.diff

messages: + msg201946
2013-11-01 19:33:38Arfreversetnosy: + Arfrever
2013-11-01 18:12:25brett.cannonlinkissue17621 dependencies
2013-10-29 05:40:31eric.snowsetfiles: + modulespec-primary-changes.diff

messages: + msg201605
2013-10-29 05:39:23eric.snowsetfiles: - modulespec-primary-changes.diff
2013-10-29 05:35:00eric.snowsetfiles: + modulespec-primary-changes.diff

messages: + msg201604
2013-10-26 21:43:54eric.snowsetfiles: - import-system-reference.diff
2013-10-26 21:43:43eric.snowsetfiles: + import-system-reference.diff

messages: + msg201399
2013-10-26 15:51:52ncoghlansetmessages: + msg201369
2013-10-25 15:37:34brett.cannonsetmessages: + msg201244
2013-10-25 08:01:51ncoghlansetmessages: + msg201224
2013-10-25 06:51:26eric.snowsetfiles: + import-system-reference.diff
2013-10-25 06:47:59eric.snowsetmessages: + msg201212
2013-10-25 06:47:12eric.snowsetfiles: - import-system-reference.diff
2013-10-25 06:40:38eric.snowsetfiles: + import-system-reference.diff
nosy: + ncoghlan
messages: + msg201210

2013-10-05 05:08:07eric.snowsetfiles: + modulespec-initial.diff
keywords: + patch
messages: + msg198981

stage: needs patch -> patch review
2013-09-27 08:22:07berker.peksagsetnosy: + berker.peksag
2013-08-28 17:36:51brett.cannonsetnosy: + brett.cannon
2013-08-28 06:56:18eric.snowcreate