If one uses the loader created by importlib.machinery.SourceFileLoader(module_name, path) in a meta path finder, where path is not a str, then the following error would happen at the moment the module is being imported. Note that, the error would not occur if the corresponding bytecode cache (__pycache__/*.pyc) is present.

  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 786, in exec_module
  File "<frozen importlib._bootstrap_external>", line 932, in get_code
  File "<frozen importlib._bootstrap_external>", line 604, in _code_to_timestamp_pyc
ValueError: unmarshallable object

Here is an example that could trigger the error. The package unipath is not very important except that unipath.Path is not marshallable. (Any library with a fancy, unmarshallable path-like object would work.) The choice of 'local' and the use of 'os.getcwd()' are also irrelevant. The actual production code is quite complicated, using different path-like objects and convoluted logic. I made up this example from scratch.

import os
import sys
import importlib
import importlib.abc
import importlib.util
from unipath import Path # just an example

class DummyFinder(importlib.abc.MetaPathFinder):
    def __init__(self):
        self.cwd = os.getcwd()
    def find_spec(self, fullname, path, target=None):
        if fullname == 'local':
            initpath = os.path.join(self.cwd, '__init__.py')
            if os.path.isfile(initpath):
                loader = importlib.machinery.SourceFileLoader(fullname, Path(initpath)) # some non-str Path-like object here
                return importlib.util.spec_from_file_location(fullname, initpath,
                        loader = loader, submodule_search_locations=[self.cwd])
        else:
            return None

sys.meta_path.append(DummyFinder())
importlib.import_module("local.test2")

If this is indeed a bug, there might be other classes and functions in importlib that share the same problem.

Maybe Brett can help?

importlib is probably not os.PathLike-clean due to its bootstrapping restrictions of not getting to use anything implemented in Python from 'os' (i.e. if it isn't being managed in posixmodule.c then it probably won't work).

If you follow the traceback it's trying to marshal a code object for the eventual .pyc file and failing (https://github.com/python/cpython/blob/faf49573963921033c608b4d2f398309d9f0d2b5/Lib/importlib/_bootstrap_external.py#L604). The real question is why is any unmarshallable object getting passed in the first place since that object is the code object that compile() returned.

Best guess? The compile() function is being given the path-like object (via https://github.com/python/cpython/blob/faf49573963921033c608b4d2f398309d9f0d2b5/Lib/importlib/_bootstrap_external.py#L848) and it's blindly setting it on the code object itself, and then marhsal fails since it can't handle pathlib.Path. If my hunch is right, then the possible solutions are:

- Don't do that 😉
- Make compile() aware of path-like objects
- Have importlib explicitly work around compile()'s shortcoming by doing the right thing for path-like objects before passing in the the 'path' argument.

If Brett's theory is right, compile() has a bug though -- it shouldn't plug a non-string into the code object. Or possibly the code object constructor should reject such objects (or extract the string).

I just filed the slightly more general bpo-45127.