This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ImportFrom level cannot be optional
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Anthony Sottile, brett.cannon, docs@python, serhiy.storchaka, srittau, thautwarm
Priority: normal Keywords:

Created on 2018-10-16 18:39 by thautwarm, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (10)
msg327840 - (view) Author: thautwarm (thautwarm) * Date: 2018-10-16 18:39
This issue is found from a type hinting problem:

```
class ImportFrom(stmt):	class ImportFrom(stmt):
    module = ...  # type: Optional[_identifier]	    module = ...  # type: Optional[_identifier]
    names = ...  # type: typing.List[alias]	    names = ...  # type: typing.List[alias]
    level = ...  # type: Optional[int]
```

As we can see that `level` here is optional, and it's strange.

I tried `ast.parse` on both Python 3.5/3.6, and found that None of `from a import *`, `from .a import *`, `from ..a import *` and other regular cases result into an ImportFrom AST whose `level` is None.


Then I went to Python-asdl: https://github.com/python/cpython/blob/137b0632dccb992ca11e9445142fb33a29c33a51/Parser/Python.asdl#L44

and got 
   
   ImportFrom(identifier? module, alias* names, int? level)


It seems like a bug. To validate it, I went to https://github.com/python/cpython/blob/97cf0828727ac2a269c89c5aa09570a69a22c83c/Python/ast.c#L3311

and got 

    static stmt_ty
    ast_for_import_stmt(struct compiling *c, const node *n){
       int idx, ndots = 0;
       ...
       return ImportFrom(modname, aliases, ndots, lineno, col_offset, c->c_arena);
       ...
    }

It seems that no reason for `level` being None.

If it's really a bug, IMO it could be also an example that type hinting helps to error detection :-)
msg327853 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2018-10-17 03:22
It appears it has always had this bug since introduction of absolute/relative imports in https://github.com/python/cpython/commit/f7f438ba3b05eb4356e7511401686b07d9dfb6d8

Agree with changing this to `# type: int` and correcting the documentation
msg327854 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2018-10-17 03:26
In fact, trying to use an `ImportFrom` without an integer `level` results in a `ValueError`:

>>> x = ast.parse('from os import path')
>>> x.body[0].level = None
>>> compile(x, '<string>', 'exec')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid integer value: None
msg327859 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-10-17 05:07
None is not integer. This field is optional:

>>> from ast import *
>>> x = Module(body=[ImportFrom(names=[alias(name='path', asname=None)], lineno=1, col_offset=0)])
>>> compile(x, '<string>', 'exec')
<code object <module> at 0x7f73b754da00, file "<string>", line 1>
msg327860 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2018-10-17 05:25
Hmmm, I don't think mypy has an annotation for "sometimes has an attribute" -- `Optional[T]` is `Union[T, None]` (why I tried `None`).

But you're right, `FromImport` is constructable without a `level` -- it seems to behave as `level=0` (I guess as expected!)
msg327869 - (view) Author: thautwarm (thautwarm) * Date: 2018-10-17 08:21
Hi, Serhiy, for Python-asdl has made `level` able to optional, certainly we could construct an ImportFrom AST without giving `level`.

Moreover, this might evidence that `level` cannot be optional in fact:

https://github.com/python/cpython/blob/8e73ad38ab7d218b9ef8976032865928dfad00f1/Python/compile.c#L2918
msg327871 - (view) Author: thautwarm (thautwarm) * Date: 2018-10-17 08:27
@Anthony

> I don't think mypy has an annotation for "sometimes has an attribute"

Yes.. The annotation for an attribute that **maybe exist** cannot be expressed by mypy..
msg327882 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-10-17 09:58
The level argument of ast.ImportFrom is optional. If it is not specified or None is passes, the corresponding attribute is set to the default value 0.

Type hints are not used in the stdlib. If this is a bug in mypy, this is not the proper tracker for reporting it.
msg327885 - (view) Author: thautwarm (thautwarm) * Date: 2018-10-17 10:10
Firstly, allowing to construct ImportFrom without `level` specified could be the result of referring Python-asdl.

Secondly, in C level, `level` is always an integer.

Last but not the least, when you can access `level` from an ImportFrom AST, it must be an integer, not None.
msg327996 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-10-18 19:07
There aren't any type hints in the 'ast' module in Python itself, so this is an issue with typeshed. Closing as "third party".
History
Date User Action Args
2022-04-11 14:59:07adminsetgithub: 79182
2018-10-18 19:07:38brett.cannonsetstatus: open -> closed

nosy: + brett.cannon
messages: + msg327996

resolution: third party
stage: resolved
2018-10-17 10:10:34thautwarmsetmessages: + msg327885
2018-10-17 09:58:09serhiy.storchakasetmessages: + msg327882
2018-10-17 08:27:08thautwarmsetmessages: + msg327871
2018-10-17 08:21:25thautwarmsetmessages: + msg327869
2018-10-17 05:25:42Anthony Sottilesetmessages: + msg327860
2018-10-17 05:07:29serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg327859
2018-10-17 03:26:42Anthony Sottilesetmessages: + msg327854
2018-10-17 03:22:03Anthony Sottilesetnosy: + Anthony Sottile
messages: + msg327853
2018-10-17 00:26:15srittausetnosy: + srittau
2018-10-16 18:39:56thautwarmcreate