On Sun, Aug 5, 2018, 08:55 Nick Coghlan, <report@bugs.python.org> wrote:

>
> Nick Coghlan <ncoghlan@gmail.com> added the comment:
>
> While I'd be inclined to agree with Paul's analysis if CPython were a
> greenfield project, I also think if SageMath had already been ported to
> Python 3.2, we would have considered this change a behavioural regression
> between 3.2 and 3.3 resulting from the importlib migration and worked out a
> way to get it to work implicitly again.
>

Perhaps, but no point on dwelling on 4 releases ago. 😉

One other thing I will say is that the PEP 302 APIs were seemingly designed
for working with Python source and were not necessarily oriented towards
alternative code representations that were executed, e.g. Quixote was
supported because it transpiled to Python source (although Paul is a
co-author on the PEP and could more definitively answer 😄). IOW the APIs
are oriented towards the Python programming language, not the CPython
interpreter. This is why, for instance, I'm suggesting that if a more
general solution is desired to be made official, then trying to force
everything into the current APIs might not turn out well.

> Now, though, we need a way for SageMath to get it working on releases up
> to and including 3.7, *without* any help from Cython or CPython, since it
> needs to work with existing Cython releases on existing CPython versions to
> avoid presenting itself to SageMath users as regression introduced by
> switching from Python 2 to Python 3.
>
> To do that, I believe it can be made to work in much the same way it did
> in Python 2 if SageMath were to do the following:
>
> 1. Define a subclass of ExtensionModuleLoader [1] that overrides
> get_source() to also look for a ".pyx" file adjacent to the extension
> module. This would also look for any of the file suffixes in
> SOURCE_SUFFIXES if a .pyx file isn't found.
> 2. Create an instance of FileFinder [3] that uses the custom loader
> subclass for any of the file suffixes in EXTENSION_SUFFIXES [4]
> 3. Replace the regular file finding hook in sys.path_hooks with  the
> path_hook method from that new FileFinder instance (by default, there's
> only one FileFinder hook installed, and it can currently be identified as
> "obj.__name__ == 'path_hook_for_FileFinder')
> 4. Invalidate importlib's caches, so any future extension module imports
> will use the custom extension module loader, rather than the default one
>

This sounds like it would work to me and doesn't require any special
changes on our part to function.

-Brett

> [1]
> https://docs.python.org/3/library/importlib.html#importlib.machinery.ExtensionFileLoader
> [2]
> https://docs.python.org/3/library/importlib.html#importlib.machinery.SOURCE_SUFFIXES
> [3]
> https://docs.python.org/3/library/importlib.html#importlib.machinery.FileFinder
> [4]
> https://docs.python.org/3/library/importlib.html#importlib.machinery.EXTENSION_SUFFIXES
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue32797>
> _______________________________________
>