In Python 2, `__path__` used to be a list, so all of the operations available to list are available, e.g., `insert`; you can also do indexing; e.g., `__path__[0]`.

However, I believe that starting from Python 3, it seems to be a <class '_frozen_importlib_external._NamespacePath'>, and a lot of operations that worked previously stopped working.It seems so abruptive and I can't find any deprecation notice on this.

Previously, one is able to insert an additional path to the front of the list by doing

module.__path__.insert(0, 'src/mypath')

Now the only procedure allowed seems to be append. Is there any way to mimic the old behaviour? Sorry, this is my first question so maybe this issue doesn't deserve any more attention. If there is any documentation that talks about the change of type of the NamespacePath, please kindly lemme know.

Sincerely,

Related issues include https://bugs.python.org/issue35843

Can you provide a short runnable example that used to work and now does not? And please show any error messages you're seeing.

Hi Eric,

Sorry for the late reply. I think I did not accurately describe the issue at all. As a minimal example, set up two virtual environments, one from 2.7.x, one from 3.7.2.

When you are in the virtual environment, do
- pip install protobuf==3.3.0
- python (to go into compiler)
- import google
- print(type(google.__path__))

The 2.7.x would tell you it's a <type 'list'>
The 3.7.2 would tell you it's a <class'_frozen_importlib_external._NamespacePath'>

Since it's not a list, some behaviours are gone, e.g., insert. I'm not trying to manipulate protobuf package's __path__, but trying to upgrade a legacy codebase, the legacy codebase does something like:

module.insert(0, 'some/path'), which breaks under python3, because it's not a list anymore, but a <class'_frozen_importlib_external._NamespacePath'>.


I believe the problem comes from the 'top level entry point'. For example, when you have 2 separate packages but they were organized under the same namespace.

E.g., in their setup.py

Package A:

    packages=[
        "orange",
        "orange.schemas",
    ],
    namespace_packages=["orange"]

Package B:
    packages=[
        "orange",
        "orange.server",
    ],
    namespace_packages=["orange"]

The problem comes from when you try to do `import orange`, and do operations with orange.__path__ here. In Python 2.7, it's a list; in Python 3.7, it's a <class '_frozen_importlib_external._NamespacePath'>

It's a _NamespacePath in 3.7 because there's no __init__.py in the top-level "google" directory, and that makes it a namespace package.

I'm not exactly sure why it works in 2.7, frankly. <time passes> Looking some more: it's because they're playing tricks in a .pth file. I don't recall what the interactions between .pth and namespace packages.

I don't know if Google really means for "google" to be a namespace package, or if it's just a side effect of how they're installing protobuf.

In any event, I don't think we want a _NamespacePath to be modifiable.

And I see you're not asking for changed behavior, just documentation. But I think it is documented that this is how namespace packages work.

Ah, that started to make sense. To be honest I didn't really pay much attention to the _namespace's documentation, I only looked at that of __path__ and somehow expected it to mention that certain namespace won't be a type<list> anymore.

I will re-read your comments and the docs for _namespace; I was also reading this https://www.python.org/dev/peps/pep-0420/#migrating-from-legacy-namespace-packages

Thanks for the help!

From the discussion is seems like there is nothing that needs to be added to the docs. I will close this in a while if nobody objects.