This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Windows: python._pth sets isolated mode late during Python initialization
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: steve.dower, vstinner
Priority: normal Keywords:

Created on 2019-02-27 14:21 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (2)
msg336760 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-02-27 14:21
The read_pth_file() of PC/getpathp.c sets Py_IsolatedFlag and Py_NoSiteFlag to 1 in Python 3.6. calculate_path() checked if a file with a "._pth" extension exists in "dllpath" or "progpath".

I refactored deeply the Python initialization in Python 3.7 and I'm not sure I introduced a regression or not.

In Python 3.7, _PyCoreConfig_Read() calls config_init_path_config() which indirectly calls read_pth_file(). pymain_read_conf_impl() detects if Py_IsolatedFlag and Py_NoSiteFlag have been modified and store the new value in cmdline->isolated and cmdline->no_site_import.

Later, cmdline_set_global_config() sets Py_IsolatedFlag and Py_NoSiteFlag; and _PyCoreConfig_SetGlobalConfig() sets Py_IgnoreEnvironmentFlag.

The problem is the relationship between isolated/cmdline.Py_IsolatedFlag, no_site_import/cmdline.Py_NoSiteFlag and Py_IgnoreEnvironmentFlag/config.ignore_environment. The isolated mode must set Py_NoSiteFlag to 1 and Py_IgnoreEnvironmentFlag to 1.

For example, pymain_read_conf_impl() uses:

    /* Set Py_IgnoreEnvironmentFlag for Py_GETENV() */
    Py_IgnoreEnvironmentFlag = config->ignore_environment || cmdline->isolated;

But it's done before calling _PyCoreConfig_Read().

Moreover, _PyCoreConfig_Read() reads PYTHONxxx environment variables before calling indirectly read_pth_file(), and so PYTHONxxx env vars are read whereas they are supposed to be ignored.

Calling read_pth_file() earlier is challenging since it depends on configuration parameters which are before calling it.

At the end, I'm not sure if it's a real issue. I'm not sure if there is a regression in Python 3.7.

--

But the code in Python 3.8 changed a lot again: _PyCoreConfig_Read() is now responsible to read all environment variables.

In Python 3.8, read_pth_file() uses a _PyPathConfig structure to set isolated and site_import parameters. These parameters are then copied to _PyCoreConfig in _PyCoreConfig_CalculatePathConfig().

Moreover, _PyCoreConfig_Read() is more explicit with the relationship between isolated, use_environment and user_site_directory. The function *starts* with:

    if (config->isolated > 0) {
        config->use_environment = 0;
        config->user_site_directory = 0;
    }

Problems (inconsistencies) arise if isolated is set from 0 to 1 after this code.
msg361400 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-02-05 02:59
Sorry I missed seeing this - I rarely browse new issues unless they're "Windows" tagged.

Reading what's in master, I don't *think* there's any regression in the main case, since the values from pathconfig override config and AFAICT they aren't used in between:

    if (pathconfig.isolated != -1) {
        config->isolated = pathconfig.isolated;
    }

Haven't looked back at 3.7, but I haven't heard of any issues coming up here from people I know who have shipped this (though they might have been using SetPath to override it all anyway).

All of this could become irrelevant though if we add support to other platforms for ._pth files :) (yes, it has been requested)
History
Date User Action Args
2022-04-11 14:59:11adminsetgithub: 80317
2021-09-21 22:27:14vstinnersetstatus: open -> closed
resolution: out of date
stage: resolved
2020-02-05 02:59:20steve.dowersetnosy: + steve.dower
messages: + msg361400
2019-02-27 14:21:26vstinnercreate