This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Embedded Python fails to import module files with version_platform extensions
Type: behavior Stage: resolved
Components: Extension Modules, Windows Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: ecosatto, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2019-04-24 22:07 by ecosatto, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (7)
msg340808 - (view) Author: Eric Cosatto (ecosatto) Date: 2019-04-24 22:07
I have an application with embedded python. When trying to import numpy, or any other module that was installed by pip, it fails with error 'ModuleNotFoundError("No module named ...")'. Yet, on the command line python, all works fine. The problem is not in the PATH, as only specific files (those with .<pyver-platform>.pyd extensions, e.g. .cp37-win_amd64.pyd) cannot be found. 

Example1: numpy

import numpy
>> ImportError("No module named 'numpy.core._multiarray_umath')

The line which fails is:
    from . import multiarray
The file it is trying to load:
C:\Program Files\Python37\Lib\site-packages\numpy\core_multiarray_umath.cp37-win_amd64.pyd


Example 2: cv2

import cv2
>> ModuleNotFoundError("No module named 'cv2.cv2'")
The line which fails is:
from .cv2 import *
The file it is trying to load:
C:\Program Files\Python37\Lib\site-packages\cv2\cv2.cp37-win_amd64.pyd
msg340809 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-04-24 22:19
Are you sure you don't have the 32-bit version of Python and the 64-bit version of those packages?
msg340810 - (view) Author: Eric Cosatto (ecosatto) Date: 2019-04-24 22:23
Yes, and note that the command line python works fine.
msg340811 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-04-24 22:39
Where are you running the embedded Python from, and how have you configured it to include the packages?

Is your install in Program Files from the full installer? Or have you extracted the embeddable distro there?
msg340846 - (view) Author: Eric Cosatto (ecosatto) Date: 2019-04-25 15:28
Hi,

I have a full install of Python from the installer in:
C:\Program Files\Python37
Then, I added libraries using pip.
Everything works fine from the command line Python.

Then, I build my C++ application with VisualStudio and link to the Python DLLs. I start Python inside my app using the usual code:

  Py_SetPath(...);
  Py_SetProgramName(...);
  Py_InitializeEx(0);

  PyRun_SimpleString("import sys");
  PyRun_SimpleString("print(sys.path)");
  PyRun_SimpleString("print(sys.prefix)");
  PyRun_SimpleString("print(sys.executable)");


>> ['C:\\Program Files\\Python37\\DLLs', 'C:\\Program Files\\Python37\\Lib', 'C:\\Program Files\\Python37', 'C:\\Program Files\\Python37\\Lib\\site-packages']
>> C:\Program Files\Python37
>> C:\Program Files\Python37\python.exe

So, everything works fine at this point. If I run the following:

  PyRun_SimpleString("import numpy");


>> ImportError("No module named 'numpy.core._multiarray_umath'")
msg341295 - (view) Author: Eric Cosatto (ecosatto) Date: 2019-05-02 18:37
Problem solved:

Running the following code in the CommandLine Python and in the C++ embedded python clarified the problem:

  import importlib.machinery
  print(importlib.machinery.all_suffixes())
  
CommandLine Python:
> ['.py', '.pyw', '.pyc', '.cp37-win_amd64.pyd', '.pyd']

C++ Embedded Python:
> ['.py', '.pyw', '.pyc', '_d.cp37-win_amd64.pyd', '_d.pyd']

It shows that the Embedded python is running in Debug mode and therefore adds a _d to the suffix. Yet the numpy or pytorch installed only non-debug files.
msg341296 - (view) Author: Eric Cosatto (ecosatto) Date: 2019-05-02 18:38
I'm closing the issue, see my previous post.
History
Date User Action Args
2022-04-11 14:59:14adminsetgithub: 80897
2019-05-02 18:38:47ecosattosetstatus: open -> closed
resolution: not a bug
messages: + msg341296

stage: resolved
2019-05-02 18:37:54ecosattosetmessages: + msg341295
2019-04-25 15:28:36ecosattosetmessages: + msg340846
2019-04-24 22:39:11steve.dowersetmessages: + msg340811
2019-04-24 22:23:14ecosattosetmessages: + msg340810
2019-04-24 22:19:26steve.dowersetmessages: + msg340809
2019-04-24 22:07:39ecosattocreate