Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: Ignore sys.modules entries with no __spec__ attribute in find_spec #80020

Open
ncoghlan opened this issue Jan 28, 2019 · 2 comments
Labels
stdlib Python modules in the Lib dir topic-importlib type-feature A feature request or enhancement

Comments

@ncoghlan
Copy link
Contributor

BPO 35839
Nosy @warsaw, @brettcannon, @ronaldoussoren, @ncoghlan, @ericsnowcurrently

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-01-28.06:54:14.206>
labels = ['type-feature', 'library']
title = 'Suggestion: Ignore sys.modules entries with no __spec__ attribute in find_spec'
updated_at = <Date 2019-02-02.01:13:43.446>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2019-02-02.01:13:43.446>
actor = 'barry'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2019-01-28.06:54:14.206>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 35839
keywords = []
message_count = 2.0
messages = ['334446', '334448']
nosy_count = 5.0
nosy_names = ['barry', 'brett.cannon', 'ronaldoussoren', 'ncoghlan', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue35839'
versions = []

@ncoghlan
Copy link
Contributor Author

(Alternate proposal inspired by the discussions in bpo-35806 and bpo-35791)

Currently, a sys.modules entry with no __spec__ attribute will prevent importlib.util.find_spec() from locating the module, requiring the following workaround:

   def find_spec_bypassing_module_cache(modname):
       _missing = object()
       module = sys.modules.pop(modname, _missing)
       try:
           spec = importlib.util.find_spec(modname)
       finally:
           if module is not _missing:
               sys.modules[modname] = module

The big downside of that approach is that it requires mutation of global state in order to work.

One of the easiest ways for this situation to be encountered is with code that replaces itself in sys.modules as a side effect of import, and doesn't bind __spec__ on the new object to the original module __spec__.

While we could take the hard line that all modules doing that need to transfer the attribute in order to be properly compatible with find_spec, I think there's a more pragmatic path we can take by differentiating between "__spec__ attribute doesn't exist" and "__spec__ attribute exists, but is None".

"__spec__ attribute doesn't exist" would be handled by find_spec as "Ignore the sys.modules entry entirely, and run the same search that would be run if the cache lookup had failed". This will then implicitly handle cases where a module replaces its own sys.modules entry.

By contrast, "__spec__ attribute is set to None" would be a true negative cache entry that indicated "this is a synthetic module that cannot be directly introspected or reloaded".

@ncoghlan ncoghlan added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Jan 28, 2019
@ncoghlan
Copy link
Contributor Author

I'd also suggest that we emit ImportWarning when taking the fallback path, since we really would prefer that sys.modules entries either have a valid spec, or else explicitly set __spec__ = None.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir topic-importlib type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants