With the code included below, and warnings enabled, I see a flood of unexpected ImportWarnings as the interpreter exits. The issue is seen with Python 3.4.0, but apparently not with 3.3.5.

The issue originally happened with code using the Py Socks library <https://github.com/Anorov/PySocks>, which I interrupted because the connect() call was hanging. I have reduced it down to the following code:

import _socket

class socket(_socket.socket):
    def __init__(self):
        _socket.socket.__init__(self)
        self.attr = self.__init__
        raise Exception()

socket()

Output from running this code:
[vadmium@localhost tmp]$ python3 -Wall script.py
/usr/lib/python3.4/site.py:333: DeprecationWarning: "site-python" directories will not be supported in 3.5 anymore
  DeprecationWarning)
Traceback (most recent call last):
  File "script.py", line 9, in <module>
    socket()
  File "script.py", line 7, in __init__
    raise Exception()
Exception
sys:1: ResourceWarning: unclosed <socket object, fd=3, family=2, type=1, proto=0>
/tmp/<frozen>:2127: ImportWarning: sys.meta_path is empty
/tmp/<frozen>:2127: ImportWarning: sys.meta_path is empty
. . .
[About two hundred lines]
. . .
/tmp/<frozen>:2127: ImportWarning: sys.meta_path is empty
/tmp/<frozen>:2127: ImportWarning: sys.meta_path is empty
[Exit 1]
[vadmium@localhost tmp]$ 

These seem to be conditions necessary to produce the issue:

* Instantiate a subclass of “_socket.socket”
* Assign a bound method to an attribute of the socket object
* Save the socket object in a local variable of a function, __init__() in my demonstration
* Raise an exception from the function holding the socket object

This might be a shutdown issue. If you print out what module is being imported at the time of the warning its 'io' and it's only coming up after the script has exited (you will notice the warnings come up after the traceback is printed). My guess is some cleanup code that gets called a lot is trying to import io after shutdown cleanup has blasted sys.meta_path.

I should also mention that if you raise an exception instead of print the warning nothing shows up, which is another indicator that this is a shutdown-related issue since that could quite easily consume the exception.

Antoine, any ideas on what might be going on?

Well, basically some sys attributes (including meta_path) are cleaned during shutdown, so if some code executes afterwards and tries to import something, importlib will complain that meta_path is empty.

Unless the warning is genuinely useful, I would suggest simply removing it.

Printing warnings during import can also degenerate into an infinite recursion, if printing the warning needs to import a module, for example:

./python -Wa -c "import sys; sys.meta_path = None; import logging"

For a more detailed analysis:
- the warning gets printed very late because the socket is caught in a reference cycle (self.attr = self), which itself is tied to the exception traceback
- when the socket is collected, it tries to print a ResourceWarning because it hasn't been closed explicitly (just add self.close() in the constructor to make it disappear)
- when trying to print the ResourceWarning, it probably wants to import linecache or something, and that breaks

The only remaining mystery is why there are 200 lines and not just one :-)

So the import warnings were added for sys.meta_path and sys.path_hooks because prior to Python 3.3 you could empty out those values and import would continue to work, but with importlib that stopped being the case. My thinking was that it would be easier to debug if you got that warning instead of your imports entirely failing and wondering what was going on. But perhaps sys.path_hooks and sys.meta_path are known well enough at this point that the warnings are superfluous. I'll open a separate bug to track that discussion.

Of course the use case presented here is very contrived, so we can also choose to not fix it.

Sure, I'm fine with closing this as "Won't Fix" based on the cause being an edge case and rely on issue #21052 to discuss the value of the ImportWarning.

Thanks for the report, Martin.

My original demonstration was distilled from a real world case. I get this issue with other real world cases too, so I am expanding the title to reflect what I think is going on. This problem seems to be caused by a ResourceWarning triggered from a garbage cycle being freed just before Python exits. The flood of ImportWarning lines makes it annoying to scroll up to see what other ResourceWarning messages, exceptions, etc you are missing out on.

Much simpler one-liner demonstration adapted from Message 222403 (because class definitions create garbage cycles in my experience):

python3 -Wall -c 'class C: a = open("/dev/null")'

The best workaround is probably to use “python -Wdefault” rather than “python -Wall”. Then you only get one ImportWarning line.

I suspect the problem might be caused by a recursive loop between emitting a warning and importing something, as Antoine hinted. Calling sys.setrecursionlimit(30) reduces the flood to 5 lines. Unfortunately my modifications to “importlib/_bootstrap.py” and “warnings.py” on my OS are not having any effect, otherwise I might try to make a patch.

Perhaps if it is not appropriate for the ImportWarning to be removed, could the code printing out the warning be changed to avoid triggering it in the first place?

I've also been affected by this when testing integration with a third-party library (NLTK). NLTK does need to be fixed, but the ResourceWarning already say so.

The new one-liner doesn't seem contrived to me.

Quentin, do you think this should be reopened?

Brett Cannon, I wonder if the only reason you closed this bug is because you thought the scenario to trigger it is very unlikely. Considering it affected someone else, and that there are other real-world triggers in addition to Py Socks, would it be okay to reopen it? I could try to make a patch or some concrete suggestions when I have a chance, if that helps change your mind :)

Perhaps a regression test could be based on this experiment:

>>> import sys, warnings
>>> sys.meta_path = None
>>> sys.modules.pop("linecache", None)
<module 'linecache' from '/usr/lib/python3.4/linecache.py'>
>>> warnings.warn("boom")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/warnings.py", line 15, in showwarning
    file.write(formatwarning(message, category, filename, lineno, line))
  File "/usr/lib/python3.4/warnings.py", line 21, in formatwarning
    import linecache
  File "/home/proj/python/lib/misc.py", line 41, in __call__
    return self.__wrapped__(name, globals, locals, fromlist, level)
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2222, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 2150, in _find_spec
  File "/usr/lib/python3.4/warnings.py", line 15, in showwarning
    file.write(formatwarning(message, category, filename, lineno, line))
. . .
  File "<frozen importlib._bootstrap>", line 2236, in _find_and_load
RuntimeError: maximum recursion depth exceeded while calling a Python object

Martin, yes, I'd be glad to see a fix if it's not too complicated.

I'm already planning to look into this problem in issue #21052 so feel free to follow over there.

This has now been fixed in https://hg.python.org/cpython/rev/d9f71bc6d897 . Thanks to everyone who helped with the bug report!