New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Raise an ImportWarning when __spec__.parent/__package__ isn't defined for a relative import #69977
Comments
When you do a relative import, package is used to help resolve it, but if it isn't defined we fall back on using name and path. We should probably consider raising an ImportWarning if package isn't defined so that some day we can consider cleaning up import and its call signature to be a bit more sane and drop the We should probably even extend it to start using __spec__.parent and then falling back to __package__, and only after both are found missing do we raise the ImportWarning about needing to use __name__ and __path__. That way __import__ can simply start taking in the spec of the calling module to do all of its work instead of having to pass all of globals(). |
Patch with tests. Not sure if importlib.h should be included? |
Including importlib.h doesn't hurt. |
that's what I figured. |
Thanks for the quick review, new patch uploaded. |
New changeset 6908b2c9a404 by Brett Cannon in branch 'default': |
Thanks for the patch, Rose! I did notice your review comments about the missing goto and the stacklevel and I simply added them myself. |
Favouring __spec__.parent over __package__ will break the documented workaround in PEP-366 for enabling explicit relative imports from __main__ even when a module is run directly instead of via -m: if __name__ == "__main__" and __package__ is None:
__package__ = "expected.package.name" It may be that workaround is something we *want* to break (as per http://bugs.python.org/issue21762#msg258332 ), but if so, it's at least worthy of a porting note in the What's New document. |
As I commented on another issue, I think I'm going to tweak the change to continue to prefer __package__, but raise ImportWarning when it doesn't match __spec__.parent. Once we do that for all attributes we can wait until Python 2.7 is done and then swap priority to __spec__ and raise a DeprecationWarning if __spec__ is missing. That way post-Python 2.7 we can move entirely to a spec-based import system. |
New changeset 219c44fe8968 by Brett Cannon in branch 'default': |
This new warning is introducing difficulties for some Cython-compiled modules: cython/cython#1720 |
Is Cython not defining actual module objects or working around types.ModuleType? I'm just trying to figure out how a Cython module is ending up in a place where the attributes that are set by PyModule_NewObject() aren't there ( cpython/Objects/moduleobject.c Line 80 in ac5bbd4
|
AFAICT, Cython simply calls PyModule_Create() on Python 3. |
OK, so the warning is triggered if __package is None or __spec__ is None ( cpython/Lib/importlib/_bootstrap.py Line 1038 in ac5bbd4
cpython/Lib/importlib/_bootstrap.py Line 1062 in ac5bbd4
Line 1520 in ac5bbd4
So the question becomes how is Cython importing modules? This warning should only be triggered if you're attempting a relative import from within a package (and thus have a level > 0). Based on Antoine's Cython issue, since something like multidict isn't a package it would suggest either Cython is setting the level > 0 when it doesn't mean to or there's a bug somewhere with level == 0 and yet Python is still trying to calculate the parent package for no reason. |
multidict is a package, the Cython module is multidict._multidict. Cython translated "import sys" into this: __pyx_t_1 = __Pyx_Import(__pyx_n_s_sys, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 5, __pyx_L1_error) And __Pyx_Import is this: Note that __Pyx_MODULE_NAME is defined thusly:
#define __Pyx_MODULE_NAME "multidict._multidict" So perhaps Cython is indeed attempting a PyImport_ImportModuleLevelObject() Why I don't know. (also since __Pyx_MODULE_NAME seems known, Cython could actually define __package__) |
Is it possible Cython is still supporting pre-PEP-328 style implicit relative imports, even in Python 2.7+? |
That might be the case. I can't find anything in the Cython docs. Stefan, could you shed a light? |
Ok, I did an experiment. I added "from __future__ import absolute_import" at the top of _multidict.c, and after a recompile the warning went away. What changed was that the following line: __pyx_t_1 = __Pyx_Import(__pyx_n_s_sys, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 5, __pyx_L1_error) was replaced with: __pyx_t_1 = __Pyx_Import(__pyx_n_s_sys, 0, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error) Ignore the change from "3" to "5", which is the line number for the traceback. The relevant change is the "level" argument to __Pyx_Import() which went from -1 to 0. |
Sorry for not responding, missed the message, it seems. Cython has to support old-style relative imports also in Py3 because that's how the user code was originally written, using Py2-style syntax and semantics. Most Cython code has not been converted to Py3 syntax/semantics, but still needs to work in Py3. From a user perspective, it's best to switch on "__future__.absolute_import" and use explicit relative imports (or write Py3 code), because it reduces the overhead at import time. I'm hoping to look into multi-step module initialisation this summer. As Nick noted, this might also help with this issue. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: