This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eryksun
Recipients brett.cannon, eryksun, mattip, paul.moore, stephtr, steve.dower, tim.golden, zach.ware
Date 2019-02-16.05:32:31
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1550295151.83.0.833008326959.issue35688@roundup.psfhosted.org>
In-reply-to
Content
It may suit the needs of NumPy and SciPy to use an assembly for DLL dependencies. With an assembly it's possible for two DLLs with the same name to load in a process and possible for a DLL to extend the assembly search path with up to nine relative paths at load time. The target directory can be up to two levels above the DLL directory (e.g. "..\..\assembly_dir"). An assembly can thus be packaged as a common dependency for other packages, and packages can depend on different versions of the assembly.

For example, let's make a package that changes the _tkinter.pyd extension module to use a private assembly, which consists of the two DLL dependencies, "tcl86t.dll" and "tk86t.dll". 

Begin by copying "DLLs\_tkinter.pyd" to a package directory such as "Lib\site-packages\mytk". Modify the embedded #2 manifest in "_tkinter.pyd" (use mt.exe, or a GUI resource editor) to include a dependency on an assembly named "amd64_tcl_tk_8.6.6.0":

  <dependency>
    <dependentAssembly>
      <assemblyIdentity name="amd64_tcl_tk_8.6.6.0"
                        version="8.6.6.0"
                        type="win32"
                        processorArchitecture="amd64" />
    </dependentAssembly>
  </dependency>

Next, add the following component configuration file beside the extension module, named "_tkinter.pyd.2.config":

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <configuration>
      <windows>
        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
          <probing privatePath="..\__winsxs__" />
        </assemblyBinding>
      </windows>
    </configuration>

This extends the assembly probing path that's used by the Fusion loader in the session server (csrss.exe). The Fusion loader probes for the assembly in four locations per directory. It checks for the assembly both as a DLL and as a manifest file, both in the directory and in a subdirectory that's named for the assembly. We'll be using a subdirectory with a manifest. 

"..\__winsxs__" resolves to "site-packages\__winsxs__". Create this directory and a subdirectory named "amd64_tcl_tk_8.6.6.0". To this, add the two DLL dependencies -- tcl86t.dll and tk86t.dll -- plus the following manifest file named "amd64_tcl_tk_8.6.6.0.manifest":

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
        <assemblyIdentity name="amd64_tcl_tk_8.6.6.0"
                          version="8.6.6.0"
                          type="win32"
                          processorArchitecture="amd64" />
        <file name="tcl86t.dll" />
        <file name="tk86t.dll" />
    </assembly>

That's all it takes. If configured properly, you should be able to import the extension module via `from mytk import _tkinter`.

This will work in a virtual environment. However, I haven't checked whether the loader handles private assemblies in the same way in a store app. That's off my radar.

> packages need to adopt to calling AddDllDirectory. As long as 
> python is built with ctypes, this is easy enough to adopt, even 
> though there are some caveats

Avoid using ctypes.windll in libraries. It caches the ctypes.WinDLL instance, which caches function pointers. Projects that use the same DLLs thus can interfere with each other by setting incompatible prototypes. It also doesn't allow us to enable use_last_error to get reliable error handling. Also, the DLL_DIRECTORY_COOKIE return value is a pointer type, not a 32-bit integer. Even if we're not using it to cleanup afterwards (i.e. AddDllDirectory; LoadLibraryExW; RemoveDllDirectory), which we should be doing, we need the full 64-bit value to reliably check for failure (NULL). By some fluke, the low DWORD of the cookie could be 0.

Here are the ctypes definitions using a private WinDLL instance and an errcheck function:

    import ctypes
    from ctypes import wintypes
    kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)

    LOAD_LIBRARY_SEARCH_DEFAULT_DIRS = 0x00001000

    DLL_DIRECTORY_COOKIE = wintypes.LPVOID

    def _errcheck_zero(result, func, args):
        if not result:
            raise ctypes.WinError(ctypes.get_last_error())
        return args

    kernel32.AddDllDirectory.errcheck = _errcheck_zero
    kernel32.AddDllDirectory.restype = DLL_DIRECTORY_COOKIE
    kernel32.AddDllDirectory.argtypes = (wintypes.LPCWSTR,)

    kernel32.RemoveDllDirectory.errcheck = _errcheck_zero
    kernel32.RemoveDllDirectory.argtypes = (DLL_DIRECTORY_COOKIE,)

    kernel32.LoadLibraryExW.errcheck = _errcheck_zero
    kernel32.LoadLibraryExW.restype = wintypes.HMODULE 
    kernel32.LoadLibraryExW.argtypes = (
        wintypes.LPCWSTR, wintypes.HANDLE, wintypes.DWORD)

Don't call SetDefaultDllDirectories. Use LoadLibraryExW(path, None, LOAD_LIBRARY_SEARCH_DEFAULT_DIRS).
History
Date User Action Args
2019-02-16 05:32:31eryksunsetrecipients: + eryksun, brett.cannon, paul.moore, tim.golden, zach.ware, mattip, steve.dower, stephtr
2019-02-16 05:32:31eryksunsetmessageid: <1550295151.83.0.833008326959.issue35688@roundup.psfhosted.org>
2019-02-16 05:32:31eryksunlinkissue35688 messages
2019-02-16 05:32:31eryksuncreate