Author cgohlke
Recipients cgohlke, paul.moore, steve.dower, tim.golden, zach.ware
Date 2015-09-08.08:05:35
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1441699536.95.0.946857245232.issue25027@psf.upfronthosting.co.za>
In-reply-to
Content
This issue was first mentioned at <http://bugs.python.org/issue24872#msg249591>.

The attached script (test_dll_load_failed.py) builds up to 256 C extension modules and imports them. On Python 3.4.3 the script passes while on Python 3.5.0rc3 the script fails with:

```
Traceback (most recent call last):
File "test_dll_load_failed.py", line 42, in <module>
import_module(name)
File "X:\Python35\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
File "<frozen importlib._bootstrap>", line 577, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 903, in create_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
```

Tested on Windows 7 and 10, 64 bit, with Python 3.5.0rc3, 32 and 64 bit.

Due to this issue the "scientific stack" is practically unusable. For example, Pandas unit tests error or abort. In a Jupyter notebook, the following simple imports fail (using current builds from <http://www.lfd.uci.edu/~gohlke/pythonlibs/>):

```
In [1]:

import matplotlib.pyplot
import pandas
import statsmodels.api
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-69fcce4de550> in <module>()
      1 import matplotlib.pyplot
      2 import pandas
----> 3 import statsmodels.api

<snip>

X:\Python35\lib\site-packages\scipy\signal\__init__.py in <module>()
    276 # The spline module (a C extension) provides:
    277 #     cspline2d, qspline2d, sepfir2d, symiirord1, symiirord2
--> 278 from .spline import *
    279 
    280 from .bsplines import *

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
```

The cause of this issue is that as of Python 3.5.0rc1 C extension modules are linked statically to the multi-threaded runtime library (/MT) instead of the multi-threaded DLL runtime library (/MD). A process can not load more than 127 statically-linked CRT DLLs using LoadLibrary due to a limit of fiber-local storage (FLS) as mentioned in the following links:

<http://stackoverflow.com/questions/1437422>
<https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/3546c3c4-1b36-4552-85c5-1b3ba860ee84>

To put the 127 limit in perspective: the pywin32 package contains 51 C extension modules, pygame 36, scipy 65, and scikit-image 41. 

In addition to C extension modules, the 127 limit also applies to statically-linked CRT DLLs that are dynamically loaded via Ctypes or LoadLibrary.
History
Date User Action Args
2015-09-08 08:05:37cgohlkesetrecipients: + cgohlke, paul.moore, tim.golden, zach.ware, steve.dower
2015-09-08 08:05:36cgohlkesetmessageid: <1441699536.95.0.946857245232.issue25027@psf.upfronthosting.co.za>
2015-09-08 08:05:36cgohlkelinkissue25027 messages
2015-09-08 08:05:35cgohlkecreate