classification
Title: official embedded Python fails to import certain modules
Type: crash Stage: resolved
Components: Documentation, Installation, Windows Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: bughunter2, eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2020-11-13 02:19 by bughunter2, last changed 2020-11-16 22:24 by bughunter2. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23322 merged steve.dower, 2020-11-16 21:14
Messages (8)
msg380867 - (view) Author: Jelle Geerts (bughunter2) Date: 2020-11-13 02:19
This problem happened with 'python-3.8.6-embed-amd64.zip' when trying to import certain modules.

Note that this problem does NOT happen with Python from 'python-3.7.9-embed-amd64.zip' (its output is also attached below).

It happened with 'import socket', 'import ctypes', and so on (see below).

Platform: Windows 7 SP1 64-bit (also KB2999226 was installed)


Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AM
D64)] on win32
>>> import lzma
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "lzma.py", line 27, in <module>
ImportError: DLL load failed while importing _lzma: De parameter is onjuist.
>>> import socket
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "socket.py", line 49, in <module>
ImportError: DLL load failed while importing _socket: De parameter is onjuist.
>>> import ctypes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "ctypes\__init__.py", line 7, in <module>
ImportError: DLL load failed while importing _ctypes: De parameter is onjuist.
>>> import bz2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "bz2.py", line 19, in <module>
ImportError: DLL load failed while importing _bz2: De parameter is onjuist.
>>>


For comparison, with Python from 'python-3.7.9-embed-amd64.zip' it does work:


Python 3.7.9 (tags/v3.7.9:13c94747c7, Aug 17 2020, 18:58:18) [MSC v.1900 64 bit
(AMD64)] on win32
>>> import lzma
>>> import socket
>>> import ctypes
>>> import bz2
>>>
msg380936 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-11-13 22:25
Sorry about the PR mess - copied the wrong issue number down, though I am looking at this one right now.

My suspicion is something in the newer vcruntime140_1.dll is to blame - does it work with Python 3.9?

You might need to update to KB3118401 [1], though I wouldn't have thought it was necessary (plus it should auto-update once you add the earlier version). It would be good to know if that's the case.

[1]: https://support.microsoft.com/kb/3118401
msg380973 - (view) Author: Jelle Geerts (bughunter2) Date: 2020-11-14 11:43
Problem still occurs with newer KB3118401 installed (instead of KB2999226).

But good news:

Installing update KB3063858 (or the older KB2533623) resolves the issue.

That update adds the new kernel32.dll functions SetDefaultDllDirectories, AddDllDirectory, and RemoveDllDirectory.

That explains why the issue didn't occur with (embedded) Python 3.7, since Python started using those functions since 3.8 if I recall correctly.

So, this issue can be considered solved. But...

Thinking along here, maybe there's a list of updates (or FAQ) to which you want to add KB3063858?

(Note of course it only affects Windows 7 users that use the embedded Python 3.8, and it doesn't affect 3.7 users and 3.9 doesn't support Windows 7.)

Or, maybe even add a bit of logic that detects when Python fails to load one of the _xxxx.pyd PE images (like _socket.pyd) and show a simple warning that tells users to check said FAQ? Or maybe that's asking too much.

Just some thoughts.
msg380989 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-11-14 20:31
> maybe there's a list of updates (or FAQ) to which you 
> want to add KB3063858?

The documentation of the embeddable package [1] could highlight the need for KB2533623. Currently it's directly mentioned only in "What's New in Python 3.8" [2].

> ImportError: DLL load failed while importing _lzma: 
> De parameter is onjuist.

Mixed-language output is awkward. A common _Py_FormatMessage function could be added in 3.10 that tries getting an error message in English before trying to get it in the user's default language.

---

[1] https://docs.python.org/3/using/windows.html#the-embeddable-package
[2] https://docs.python.org/3.8/whatsnew/3.8.html#changes-in-the-python-api
msg380990 - (view) Author: Jelle Geerts (bughunter2) Date: 2020-11-14 20:56
> Mixed-language output is awkward. A common _Py_FormatMessage function could be added in 3.10 that tries getting an error message in English before trying to get it in the user's default language.

I agree that in general mixed-language can be odd. But I guess you could also argue it's useful that Python doesn't add another layer of complexity, and simply uses the system's own FormatMessage() function, also because developers are already used to seeing such messages in other applications as well. Just playing devil's advocate here.

At the same time, I feel it would be helpful to users if there would, in this specific case, be a more helpful message as well (alongside the same error that is already shown), something that points more directly in the direction of the solution.

> The documentation of the embeddable package [1] could highlight the need for KB2533623. Currently it's directly mentioned only in "What's New in Python 3.8" [2].

That would be helpful.
msg381162 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-11-16 21:28
New changeset fa86614078f1f700f350f98486dddf83587ab69e by Steve Dower in branch '3.8':
bpo-42339: Adds note about KB2533623 to embeddable package docs (GH-23322)
https://github.com/python/cpython/commit/fa86614078f1f700f350f98486dddf83587ab69e
msg381163 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-11-16 21:29
I added a note to the 3.8 docs.

Pretty sure there's another open issue about improving FormatMessage, and if not, one can be opened.

Thanks for the report, Jelle, and for helping us figure it out!
msg381171 - (view) Author: Jelle Geerts (bughunter2) Date: 2020-11-16 22:24
Thanks to you too, Steve! *tips imaginary hat*
History
Date User Action Args
2020-11-16 22:24:17bughunter2setmessages: + msg381171
2020-11-16 21:29:23steve.dowersetstatus: open -> closed
messages: + msg381163

components: + Documentation
resolution: fixed
stage: patch review -> resolved
2020-11-16 21:28:39steve.dowersetmessages: + msg381162
2020-11-16 21:14:42steve.dowersetpull_requests: + pull_request22214
2020-11-14 20:56:57bughunter2setmessages: + msg380990
2020-11-14 20:31:55eryksunsetnosy: + eryksun
messages: + msg380989
2020-11-14 11:43:44bughunter2setmessages: + msg380973
2020-11-13 22:25:06steve.dowersetmessages: + msg380936
2020-11-13 22:18:27steve.dowersetpull_requests: - pull_request22167
2020-11-13 21:54:13steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request22167
2020-11-13 07:49:14christian.heimessetassignee: steve.dower

components: + Windows
nosy: + paul.moore, tim.golden, steve.dower, zach.ware
2020-11-13 02:19:41bughunter2create