Author pje
Recipients barry, brett.cannon, eric.snow, inada.naoki, kristjan.jonsson, pconnell, pje
Date 2013-04-05.17:23:40
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CALeMXf5_Z=h_oNRAKsvuoF_Zbes8n1YYDFZdEPyDGXwMTEggxA@mail.gmail.com>
In-reply-to <1365167262.93.0.0293704343025.issue17636@psf.upfronthosting.co.za>
Content
On Fri, Apr 5, 2013 at 9:07 AM, Kristján Valur Jónsson
<report@bugs.python.org> wrote:
> But I can think of contrived examples where this could break things:
> #a.py:
> from .b import util
>
> #b.py
> from . import a
> from .util import util
>
> Because of the circular import order, b.util will not exist as an attribute on b when a does its import.  So a.util will end up being util instead of util.util as one might expect.

Not quite.  It will only do this if the '.b.util' module is *in
sys.modules* at the time that a is being imported, which also means
that .b.util has to be imported somewhere *before* .a, in order to end
up with a stale value. As written, your example actually works
correctly if .a is imported first, and fails with an ImportError if .b
is imported first.

In fact, this example is kind of useful for proving the change
*correct*, not broken.   At the very least, it shows that you'll have
to be more inventive to come up with a breaking case.  ;-)

Consider that for any module x.y, x must be in sys.modules before x.y
can.  Therefore, any "from x import" taking place before x is fully
loaded will either happen before x.y is fully loaded, during the load,
or after it, and the following cases apply:

1. If it happens before, then it fails with an ImportError as is the case today.
2. If it happens during (i.e. there is a cycle with x.y rather than
with just x),
   then the import returns the module.
3. If it happens after, then either the x.y attribute is bound to the submodule,
   or has been rebound to something else, or deleted.
4. If after and deleted, the import returns the module.
5. If after and rebound, the import returns the changed attribute
(just like today)
6. If after and normally bound, the import returns the module (just like today)

The only cases in which the behavior changes from today are cases 2
and 4, which would both fail today with an ImportError.  Case 4 also
doesn't make much sense, since 'import x.y' would still permit access
to the module -- so it'd have to be an odd situation in which you
didn't want 'from import' (and *only* from import) to fail.

So let's consider case 2, which would have to be written something like:

#a.py
from .b import util

#b.py
from .util import util

#b/util.py
from .. import a
def util(): pass

#__main__.py
import b

So, import b leads to starting the load of b.util, which leads to
importing a, which succeeds and sees the b.util module instead of the
b.util:util function.

But, because of the circularity, this will *also* happen if you import
a first. So 'a' will *always* see b.util rather than b.util:util,
unless you remove the circularity.  If you remove the circularity,
then 'a' will always see b.util:util as the value of util.

So case 2 does not lead to a hard-to-debug ordering dependency, it
leads to an immediately discoverable change in behavior the moment you
start importing a from b.util.

The tl;dr version of the above is that you will only receive a module
instead of an attribute if the module that's about to be replaced just
imported *you* during its initial loading, and if it does that, it'll
do it no matter what order the imports occur in, making the problem
occur immediately and consistently as soon as you introduce the
circularity.
History
Date User Action Args
2013-04-05 17:23:41pjesetrecipients: + pje, barry, brett.cannon, kristjan.jonsson, inada.naoki, eric.snow, pconnell
2013-04-05 17:23:41pjelinkissue17636 messages
2013-04-05 17:23:40pjecreate