classification
Title: Re-use already existing functionality to allow Python 2.7.x (both embedded and standalone) to locate the module path according to the shared library
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Ali Rizvi-Santiago, vstinner
Priority: normal Keywords: patch

Created on 2018-11-06 00:18 by Ali Rizvi-Santiago, last changed 2019-10-04 18:52 by Ali Rizvi-Santiago. This issue is now closed.

Files
File name Uploaded Description Edit
relocate-with-dllpath.patch Ali Rizvi-Santiago, 2018-11-06 00:18 Patch to PC/getpathp.c to add usage of dllpath static variable
Messages (3)
msg329322 - (view) Author: Ali Rizvi-Santiago (Ali Rizvi-Santiago) * Date: 2018-11-06 00:18
This is specific to the Windows platform as it's the only platform that uses the registry and other path hacks to identify the default module path. This patch is mostly intended for embedded Python, but is also applicable to a stand-alone Python.

A few years ago, I was looking at relocating my Python interpreter so that an embedded application (that I didn't have control over) would use the correct module path. While merging my code into CPython, I quickly noticed that Python already supported doing this with Py_ENABLE_SHARED but due to the implementation wasn't actually using it for some reason. The code that implements this is 10+ years old, so perhaps it was just an oversight or some other reason that I didn't know about.

Inside PC/getpathp.c there's a static variable, "dllpath", that is initialized with the path to the DLL that is being dynamically loaded when Py_ENABLE_SHARED is specified. Normally arg0 is used to locate the module path, but when embedding Python the .exe and thus arg0 is not involved. So, this patch uses "dllpath" by adding a final case to the calculation of the path by assigning it to "pythonhome" if the home was not able to be determined by any other means. This is done in 2 places within "calculatepath()".

This allows one to have multiple versions of Python (32-bit, 64-bit, older versions, etc) on the same Windows system and so a user should not need to copy the Python library into their System path or explicitly set any environment variables (unless truly desired of course). This should greatly simplify relocation of Python as the DLL and executable can be moved around without being dependant on any external invariants.
msg353249 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 00:59
Modules/getpath.c and PC/getpathp.c have been deeply reworked with the implementation of the PEP 587 in Python 3.8. Parameters which are already set (ex: Py_SetProgramName() or using the new PyConfig API) are no longer overriden. I added some tests on the "path configuration".

The most reliable way to configure the path configuration is now to use the new initialization API using PyConfig.
https://docs.python.org/dev/c-api/init_config.html
msg353970 - (view) Author: Ali Rizvi-Santiago (Ali Rizvi-Santiago) * Date: 2019-10-04 18:52
Sure. It was for Python2 anyways.

-Ali

On Wed, Sep 25, 2019 at 8:00 PM STINNER Victor <report@bugs.python.org>
wrote:

>
> STINNER Victor <vstinner@python.org> added the comment:
>
> Modules/getpath.c and PC/getpathp.c have been deeply reworked with the
> implementation of the PEP 587 in Python 3.8. Parameters which are already
> set (ex: Py_SetProgramName() or using the new PyConfig API) are no longer
> overriden. I added some tests on the "path configuration".
>
> The most reliable way to configure the path configuration is now to use
> the new initialization API using PyConfig.
> https://docs.python.org/dev/c-api/init_config.html
>
> ----------
> nosy: +vstinner
> resolution:  -> fixed
> stage:  -> resolved
> status: open -> closed
> versions: +Python 3.8, Python 3.9 -Python 2.7
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue35173>
> _______________________________________
>
History
Date User Action Args
2019-10-04 18:52:14Ali Rizvi-Santiagosetmessages: + msg353970
2019-09-26 00:59:57vstinnersetstatus: open -> closed

versions: + Python 3.8, Python 3.9, - Python 2.7
nosy: + vstinner

messages: + msg353249
resolution: fixed
stage: resolved
2018-11-06 00:26:49Ali Rizvi-Santiagosettitle: Re-use already existing functionality to allow Python 2.7.x (both embedded and standalone) to locate the module path -> Re-use already existing functionality to allow Python 2.7.x (both embedded and standalone) to locate the module path according to the shared library
2018-11-06 00:18:27Ali Rizvi-Santiagocreate