classification
Title: venv does not install libpython
Type: behavior Stage: resolved
Components: C API, Installation, Library (Lib) Versions:
process
Status: open Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: AliyevH, anuppari, christian.heimes, vinay.sajip
Priority: normal Keywords:

Created on 2021-02-26 22:40 by anuppari, last changed 2021-03-16 21:47 by christian.heimes.

Messages (8)
msg387752 - (view) Author: Anup Parikh (anuppari) Date: 2021-02-26 22:40
The libpython.a or libpython.so objects are not copied/linked in new virtual environments when created with venv.
msg388494 - (view) Author: Hasan (AliyevH) * Date: 2021-03-11 10:10
Can you please provide more information about this behavior?
msg388552 - (view) Author: Anup Parikh (anuppari) Date: 2021-03-12 21:09
A typical python installation includes libpython libraries that C extensions can link to. Creating virtual environments with venv normally creates a replica of a python installation directory, including all the built in python modules. However, it doesn't seem to copy over the libpython libraries, so C extensions built with a venv activated can't seem to find the libpython (.so, .a, .dll) file to link against.

For example, compare the installation directory of python with the venv directory. The libpython .so/.a/.dll files are missing in the venv
msg388553 - (view) Author: Anup Parikh (anuppari) Date: 2021-03-12 21:11
Also, at least on windows, the C-API header files are also missing from the venv
msg388720 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-03-15 09:57
This is not a bug - venvs are primarily *run*-time (as opposed to development-time) environments, into which you install pre-built Python packages (including ones with C extensions). Can you give a specific example where creating a venv and installing packages into it leads to a problem due to libpython.XXX being unavailable, with a series of steps to reproduce? If not, this issue will need to be closed.
msg388724 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-03-15 10:03
I agree with Vinay. venvs don't contain copies of libpython or header files by design. setuptools will pcik them up from the main installation.

If you have any issues with compiling C extensions, please report them with setuptools at https://github.com/pypa/setuptools/
msg388883 - (view) Author: Anup Parikh (anuppari) Date: 2021-03-16 21:08
I'm seeing this issue in a third-party package that uses CMake and Make based build systems rather then setuptools (it's a mostly C library which additionally provides some python wrappers). Those build systems can normally parse a python installation with the standard directory structure to find the library and headers, but fail on a venv. I can bring this up with the package developers, but I don't see any downsides to linking the headers/libraries in the venv to make it easy to use.

Regarding the comment that venv's are primarily for runtime, I didn't see anything in PEP 405 that supports that statement. And in any case, even for runtime environments, installing a python package from a source distribution is valid, right? So shouldn't the venv support building extensions?

Again, I agree that third party packages that provide python wrappers should probably use standard tools (i.e., setuptools) to build the python C extensions, but I fail to see the downsides to including links to the headers/libraries in the venv for ease of use.
msg388889 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-03-16 21:47
This sounds like a bug in CMake or Make. Are you using any CMake plugins or autoconf/automake macros? It's very well possible that the author of these extension made a wrong assumption or the extension was written before venvs were introduced.
History
Date User Action Args
2021-03-16 21:47:22christian.heimessetmessages: + msg388889
2021-03-16 21:08:54anupparisetstatus: closed -> open

messages: + msg388883
2021-03-15 10:03:34christian.heimessetstatus: pending -> closed

nosy: + christian.heimes
messages: + msg388724

resolution: not a bug
stage: resolved
2021-03-15 09:57:59vinay.sajipsetstatus: open -> pending

messages: + msg388720
2021-03-14 04:41:28ned.deilysetnosy: + vinay.sajip
2021-03-12 21:11:30anupparisetmessages: + msg388553
2021-03-12 21:09:58anupparisetmessages: + msg388552
2021-03-11 10:10:02AliyevHsetnosy: + AliyevH
messages: + msg388494
2021-02-26 22:40:16anupparicreate