classification
Title: PathFinder is twice on sys.meta_path
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.snow, htgoebel, miss-islington, ncoghlan, ned.deily, pablogsal
Priority: normal Keywords: 3.7regression, patch

Created on 2018-03-23 23:30 by htgoebel, last changed 2018-04-25 02:58 by ncoghlan. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6273 merged pablogsal, 2018-03-27 22:35
PR 6592 merged miss-islington, 2018-04-25 02:22
Messages (8)
msg314340 - (view) Author: Hartmut Goebel (htgoebel) Date: 2018-03-23 23:30
As of Python 3.7.0b2 _frozen_importlib_external.PathFinder exists twice on sys.meta_path, and it is the same object:

$ python -S
Python 3.7.0b2 (default, Mar 22 2018, 20:09:00) 
[GCC 5.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.meta_path)
[<class '_frozen_importlib.BuiltinImporter'>,
<class '_frozen_importlib.FrozenImporter'>,
<class '_frozen_importlib_external.PathFinder'>,
<class '_frozen_importlib_external.PathFinder'>]
>>> print([id(p) for p in sys.meta_path])
[24427944, 24430216, 24517416, 24517416]
>>>
msg314348 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-03-24 01:01
Thanks for the report.  git bisect result:

1abcf6700b4da6207fe859de40c6c1bada6b4fec is the first bad commit
commit 1abcf6700b4da6207fe859de40c6c1bada6b4fec
Author: Eric Snow <ericsnowcurrently@gmail.com>
Date:   Tue May 23 21:46:51 2017 -0700

    bpo-22257: Private C-API for core runtime initialization (PEP 432). (#1772)

    (patch by Nick Coghlan)
msg314556 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-03-27 22:21
It seems that the problem is that `_Py_InitializeEx_Private` calls  `_Py_InitializeCore` and `_Py_InitializeMainInterpreter`. The first one calls at the end `initimport` that in turns calls `importlib._install_external_importers` that sets the PathFinder. Then, `_Py_InitializeMainInterpreter` calls  `initexternalimport` that in turns calls again `importlib._install_external_importers` and therefore we end with two PathFinders in sys.meta_path.

I think the solution is removing the call to initexternalimport in `_Py_InitializeMainInterpreter` as it has been alreade initialized by `_Py_InitializeCore`.
msg314557 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-03-27 22:33
The same problem happens in `new_interpreter` as far as I understand.
msg314628 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-03-29 01:50
Calling initexternalimports InitalizeMainInterpreter and from new_interpreter is right, since we only want these importers on the metapath after we know sys.path has been configured appropriately.

It's the call from InitializeCore that's questionable, since we haven't finished setting up sys.path at that point.
msg315717 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-25 02:22
New changeset 0977091dca59709864b14cfc129388f1f0de7bf7 by Nick Coghlan (Pablo Galindo) in branch 'master':
bpo-33128 Fix duplicated call to importlib._install_external_importers  (GH-6273)
https://github.com/python/cpython/commit/0977091dca59709864b14cfc129388f1f0de7bf7
msg315718 - (view) Author: miss-islington (miss-islington) Date: 2018-04-25 02:48
New changeset 52a5a17338dfa7fed259027e1ecceba9c8491189 by Miss Islington (bot) in branch '3.7':
bpo-33128 Fix duplicated call to importlib._install_external_importers  (GH-6273)
https://github.com/python/cpython/commit/52a5a17338dfa7fed259027e1ecceba9c8491189
msg315719 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-25 02:58
Hartmut, thanks for the issue report, and Pablo, thanks for the PR to resolve it!
History
Date User Action Args
2018-04-25 02:58:44ncoghlansetstatus: open -> closed
resolution: fixed
messages: + msg315719

stage: patch review -> resolved
2018-04-25 02:48:07miss-islingtonsetnosy: + miss-islington
messages: + msg315718
2018-04-25 02:22:52miss-islingtonsetpull_requests: + pull_request6290
2018-04-25 02:22:35ncoghlansetmessages: + msg315717
2018-03-29 01:50:32ncoghlansetmessages: + msg314628
2018-03-27 22:35:59pablogsalsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request6001
2018-03-27 22:33:56pablogsalsetmessages: + msg314557
2018-03-27 22:21:59pablogsalsetnosy: + pablogsal
messages: + msg314556
2018-03-26 19:47:01brett.cannonsetkeywords: + 3.7regression
nosy: + brett.cannon
2018-03-24 01:01:39ned.deilysetversions: + Python 3.8
nosy: + eric.snow, ncoghlan, ned.deily

messages: + msg314348

stage: needs patch
2018-03-23 23:30:41htgoebelcreate