When running Python inside PostgreSQL using plpython on OSX 10.7.5 I started coming across very strange and apparently random ImportErrors. For example, failing to find the stat module while importing site:

     Traceback (most recent call last):
       File "/Users/jinty/.buildout/eggs/setuptools-0.6c11-py2.7.egg/site.py", line 73, in <module>
         __boot()
       File "/Users/jinty/.buildout/eggs/setuptools-0.6c11-py2.7.egg/site.py", line 2, in __boot
         import sys, imp, os, os.path
       File "/Users/jinty/src/mp/lib/python2.7/os.py", line 49, in <module>
         import posixpath as path
       File "/Users/jinty/src/mp/lib/python2.7/posixpath.py", line 15, in <module>
         import stat
     ImportError: No module named stat

I debugged this by using PYTHONVERBOSE and modifying import.c with the attached patch. I found that stat.py was not found because fopen() failed with "Too many open files". There were not enough open files because OSX has insanely low limits and PostgreSQL was using a large chunk of that. ulimit -n 4096 resolved the errors. I spent a LOT of time trying to figure that out (see the thread at http://www.postgresql.org/message-id/20130114163014.GA600@Brians-MacBook-Air.local).

The bug I wish to report is that the real error (Too many open files) is hidden by "ImportError: No module named stat". For anyone who does not want to modify import.c and rebuild python, it is almost impossible to figure out what is really happening.

I wonder if importlib replicates this behavior, it may need fixing as well.

A quick trace through importlib._bootstrap through hg.python.org would suggest that the OSError would propagate when accessing source (OSError is swallowed when you try and write bytecode, but that's legitimate semantics).

Now that importlib is import and it would raise OSError I'm closing this as fixed.