classification
Title: typing forward references and module attributes
Type: Stage:
Components: Library (Lib) Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alex.gronholm, gvanrossum, levkivskyi, mjpieters, ztane
Priority: normal Keywords:

Created on 2016-03-03 18:07 by mjpieters, last changed 2016-09-27 22:26 by gvanrossum. This issue is now closed.

Messages (7)
msg261172 - (view) Author: Martijn Pieters (mjpieters) * Date: 2016-03-03 18:07
Forward references to a module can fail, if the module doesn't yet have the required object. The "forward references" section names circular dependencies as one use for forward references, but the following example fails:

$ cat test/__init__.py
from .a import A
from .b import B
$ cat test/a.py
import typing
from . import b

class A:
    def foo(self: 'A', bar: typing.Union['b.B', None]):
        pass
$ cat test/b.py
import typing
from . import a

class B:
    def spam(self: 'B', eggs: typing.Union['a.A', None]):
        pass
$  bin/python -c 'import test'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/__init__.py", line 1, in <module>
    from .a import A
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/a.py", line 2, in <module>
    from . import b
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/b.py", line 4, in <module>
    class B:
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/b.py", line 5, in B
    def spam(self: 'B', eggs: typing.Union['a.A', None]):
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 537, in __getitem__
    dict(self.__dict__), parameters, _root=True)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 494, in __new__
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 494, in <genexpr>
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 185, in __subclasscheck__
    self._eval_type(globalns, localns)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 172, in _eval_type
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
AttributeError: module 'test.a' has no attribute 'A'

The forward reference test fails because only NameError exceptions are caught, not AttributeError exceptions.
msg261173 - (view) Author: Martijn Pieters (mjpieters) * Date: 2016-03-03 18:11
Sorry, that should have read "the forward references section of PEP 484".

The section uses this example:

# File models/a.py
from models import b
class A(Model):
    def foo(self, b: 'b.B'): ...

# File models/b.py
from models import a
class B(Model):
    def bar(self, a: 'a.A'): ...

# File main.py
from models.a import A
from models.b import B

which doesn't fail because the forward references are not being tested until after all imports have completed; creating a Union however triggers a subclass test between the different types in the union.
msg261174 - (view) Author: Martijn Pieters (mjpieters) * Date: 2016-03-03 18:17
A temporary work-around is to use a function to raise a NameError exception when the module attribute doesn't exist yet:

def _forward_A_reference():
    try:
        return a.A
    except AttributeError:
        # not yet..
        raise NameError('A')

class B:
    def spam(self: 'B', eggs: typing.Union['_forward_A_reference()', None]):
        pass
msg261175 - (view) Author: Alex Grönholm (alex.gronholm) * Date: 2016-03-03 18:24
I wonder why they forward references are evaluated *at all* at this point. Seems senseless to me. This should be the job of the static type checker or the get_type_hints() function.
msg261176 - (view) Author: Antti Haapala (ztane) * Date: 2016-03-03 18:30
Indeed, the assumption is be that if a string is used, it is used there because the actual thing cannot be referenced by name at that point. Then trying to evaluate it at all would be an optimization in only those cases where it is used incorrectly / needlessly.
msg261177 - (view) Author: Martijn Pieters (mjpieters) * Date: 2016-03-03 18:38
> I wonder why they forward references are evaluated *at all* at this point. 

The Union type tries to reduce the set of allowed types by removing any subclasses (so Union[int, bool] becomes Union[int] only). That's all fine, but it should not at that point fail if a forward reference is not available yet.

Arguably, the except NameError there should be converted to a except Exception, since forward references are supposed to be *a valid Python expression [...] and it should evaluate without errors once the module has been fully loaded.* (from the PEP); anything goes, and thus any error goes until the module is loaded.
msg277561 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-09-27 22:26
Fixed by 09cc43df4509.
History
Date User Action Args
2016-09-27 22:26:39gvanrossumsetstatus: open -> closed

nosy: + gvanrossum
messages: + msg277561

resolution: fixed
2016-06-30 22:04:19levkivskyisetnosy: + levkivskyi
2016-03-03 18:38:13mjpieterssetmessages: + msg261177
2016-03-03 18:30:50ztanesetnosy: + ztane
messages: + msg261176
2016-03-03 18:24:18alex.gronholmsetnosy: + alex.gronholm
messages: + msg261175
2016-03-03 18:17:39mjpieterssetmessages: + msg261174
2016-03-03 18:11:39mjpieterssetmessages: + msg261173
2016-03-03 18:07:47mjpieterscreate