This issue was first mentioned at <http://bugs.python.org/issue24872#msg249591\>.

The attached script (test_dll_load_failed.py) builds up to 256 C extension modules and imports them. On Python 3.4.3 the script passes while on Python 3.5.0rc3 the script fails with:

Traceback (most recent call last):
File "test_dll_load_failed.py", line 42, in <module>
import_module(name)
File "X:\Python35\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
File "<frozen importlib._bootstrap>", line 577, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 903, in create_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

Tested on Windows 7 and 10, 64 bit, with Python 3.5.0rc3, 32 and 64 bit.

Due to this issue the "scientific stack" is practically unusable. For example, Pandas unit tests error or abort. In a Jupyter notebook, the following simple imports fail (using current builds from <http://www.lfd.uci.edu/~gohlke/pythonlibs/\>):

In [1]:

import matplotlib.pyplot
import pandas
import statsmodels.api
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-69fcce4de550> in <module>()
      1 import matplotlib.pyplot
      2 import pandas
----> 3 import statsmodels.api

<snip>

X:\Python35\lib\site-packages\scipy\signal\__init__.py in <module>()
    276 # The spline module (a C extension) provides:
    277 #     cspline2d, qspline2d, sepfir2d, symiirord1, symiirord2
--> 278 from .spline import *
    279 
    280 from .bsplines import *

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

The cause of this issue is that as of Python 3.5.0rc1 C extension modules are linked statically to the multi-threaded runtime library (/MT) instead of the multi-threaded DLL runtime library (/MD). A process can not load more than 127 statically-linked CRT DLLs using LoadLibrary due to a limit of fiber-local storage (FLS) as mentioned in the following links:

<http://stackoverflow.com/questions/1437422\>
<https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/3546c3c4-1b36-4552-85c5-1b3ba860ee84\>

To put the 127 limit in perspective: the pywin32 package contains 51 C extension modules, pygame 36, scipy 65, and scikit-image 41.

In addition to C extension modules, the 127 limit also applies to statically-linked CRT DLLs that are dynamically loaded via Ctypes or LoadLibrary.

If the scientific stack is unusable, I think this should be a release
blocker.

This is your wheelhouse, Steve.

Let me experiment today with a few of the proposals I posted in the other thread and get back to you.

I suspect someone will need to ship vcruntime.dll, and I'd rather it was the extension.

Is Python-core built with /MD? I cannot see the flags in the buildbot
logs.

Kind-of... We use the same flags I described in my blog1 so that we don't have any version-specific dependencies.

You should (might) see /MT in the build logs, but then we replace most of the static CRT with the dynamic (but versionless) one. The versioned parts (including the FlsAlloc call - module initialization is compiler version specific) are statically linked.

I'm going to try and update distutils to build with /MD again (sorry Christoph!) and include vcruntime###.dll in the output. That way, people who bdist_wheel will include all of their own dependencies and don't have to worry about whether users are on Python 3.5.0 or 3.9.9.

It seems to be /MTd. Sorry for the noise (and yay! for horizontal scrolling :).

The reason I asked: We had issues where extension modules linked
against a different CRT had isolated locale settings from the
interpreter, i.e., changes made by the module would not be seen
by the interpreter.

I don't know if this is still an issue with the new runtimes.

It shouldn't be - locale state is in the shared part of the CRT. That is one of the reasons I was keen to move to this model (everyone seems to think that FILE* is the only problem with mixing CRT versions :) )

Attached a fix for distutils that will include the required vcruntime DLL with the extension.

This is purely Python source code and only needs to be patched on the build machine.

I have tested with a numpy build from source (setup.py bdist_wheel) and it works correctly on a clean Win7 machine with only a Python 3.5.0rc3 install.

When multiple vcruntime###.dll files are available, the first one that is loaded wins. This means that if you can guarantee a particular import occurs first, you can omit the DLL elsewhere in your package. There is no way to determine this automatically,

Because of the way I wrote the patch, if you build with DISTUTILS_USE_SDK set, you will get the old static linking unless you also define PY_VCRUNTIME_REDIST to the path of the file to include. If the redist file cannot be found (you probably don't have the compiler either, but assuming you do), you will get the old static linking. I think this is the right balance of "works-by-default" and "I know what I'm doing let me control it" (though now I put them next to each other, I could rename the variable to DISTUTILS_VCRUNTIME_REDIST).

Will work up a test for it, but wanted to get feedback on the approach first.

I understand that distributing dependent DLLs next to extension modules is considered the best approach <https://mail.python.org/pipermail/distutils-sig/2014-October/024990.html\> (which nevertheless fails in common cases), however vcruntime140.dll is a special case since it will be shared by almost all extension modules and can be considered a system library.

My Python 3.4 installation contains 913 .pyd files in 277 directories under Lib\site-packages. With the proposed change there will be ~277 redundant vcruntime140.dll files under Python 3.5. Size is not an issue since vcruntime140.dll is small (~87 KB for 64 bit).

Many extension modules are installed directly into Lib\site-packages (no package directory). Uninstalling any one of those extension modules using pip or "wininstaller" will delete vcruntime140.dll from Lib\site-packages, potentially breaking the other extension modules in Lib\site-packages.

IANAL, but under GPL "you may not distribute these [runtime] libraries in compiled DLL form with the program" <http://www.gnu.org/licenses/gpl-faq.html#WindowsRuntimeAndGPL\>.

vcruntime140.dll *is* a system library when installed properly, and if someone installs VCRedist then all the bundled ones should be ignored. Over time, I expect to see extensions appear that depend on vcruntime150.dll rather than 140.dll, so it won't always be shared by all extensions.

However, if someone has vcruntime140.dll installed and an extension requires vcruntime150.dll, they will get errors unless that extension includes the correct version. We can't ship currently-nonexistent versions with Python 3.5, and if extensions have to depend on what's installed then Python 3.5 extensions will forever be tied to MSVC 14.0.

GPL code should either statically link or recommend their users install VCRedist separately. I'm not going to compromise compiler version independence because of one license.

The uninstall issue is something I hadn't considered. Maybe we can special-case pip uninstalling it from the site-packages folder? *Paul* - any thoughts?

I think 3.5 should be distributed with vcruntime140.dll. It seems a waste for python.exe, python35.dll, and all of the extension modules and dependent DLLs to each take an FLS slot:

    >>> import ctypes # +1 for _ctypes
    >>> kernel32 = ctypes.WinDLL('kernel32')
    >>> kernel32.FlsGetValue.restype = ctypes.c_void_p
    >>> [x for x in range(128) if kernel32.FlsGetValue(x)]
    [1, 2, 4, 5, 6, 8]

    >>> import pyexpat, select, unicodedata, winsound
    >>> import _bz2, _decimal, _elementtree, _hashlib
    >>> import _lzma, _msi, _multiprocessing, _overlapped 
    >>> import _socket, _sqlite3, _ssl, _tkinter
    >>> [x for x in range(128) if kernel32.FlsGetValue(x)]
    [1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27]

Okay, here's a proposal:

We bundle vcruntime140.dll with Python's normal install, so it's always there and extensions that use it do not need to ship anything.

When distutils._msvccompiler is used to build an extension with a *different* version of MSVC, it will copy the dependency or statically link (as in my attached patch).

This does not prevent us from changing the compiler used for 3.5, as long as we continue to ship both vcruntime140.dll and the newer version, and extensions build with newer compilers will include the dependency or pick up the bundled one if they are on a version that includes it.

(Extensions that use C++ and depend on msvcp###.dll will need to ship that themselves, obviously.)

I'll post a new patch shortly, but it's only a very small change from this one for distutils, and the rest is in the installer.

New patch. Mostly build and installer changes, but the distutils/_msvccompiler.py is also part of it.

I've run a full build and done basic testing with a full test run going now, but I don't have a clean machine handy to try it without the full CRT install, so that'll have to wait until tomorrow.

This *basically* reverts the build back to /MD for everything. The one exception is that distutils now knows which DLLs are shipped with Python and if a vcruntime is needed that isn't included, it will be put into the dist (or statically linked). So wheels created with 3.5.6 and MSVC 15.0 will still run against 3.5.0, even if the user has not installed the latest VCRedist.

FYI: we're making a new release (right now!) with the patch applied, that should go out tomorrow.

If anyone spots anything important in the patch, I still really want to hear about it, but hopefully having something installable means we'll get at least a few days of testing before locking it in.

All my apologies for waiting until the last minute before giving up on the crusade to avoid including the versioned files in Python. The timing is unfortunate, but I'm sure we're going to have the best compatibility story possible right now. So thanks for indulging me, and I hope I haven't put too many people through too much anguish.

Maybe we can special-case pip uninstalling it from the site-packages folder? *Paul* - any thoughts?

Sorry, I've been following this thread but it's been moving pretty fast, so I'm probably replying too late to be helpful now :-(

One alternative thought I had was to bundle vcruntime140.dll in a separate wheel, which other wheels can depend on. Then we get pip's usual dependency resolution to handle ensuring that the runtime is present.

It's possible to special-case vcruntime, what I'd do is modify distutils to omit vcruntime140.dll from the RECORD file - then nothing (pip, distlib, other install tools) views the vcruntime file as being "owned" by a particular package. I'm not overly keen on that solution, though, as it's clearly a hack and would be a maintainability issue going forward. But it does keep the code changes isolated to the core, rather than needing pip/setuptools changes.

New changeset 8374472c6a6e by Steve Dower in branch '3.5':
Issue bpo-25027: Reverts partial-static build options and adds vcruntime140.dll to Windows installation.
https://hg.python.org/cpython/rev/8374472c6a6e

Pull request accepted. Please forward-merge, thanks!