classification
Title: Py_Initialize(): computing path configuration must not have side effect (PEP 432)
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: vstinner
Priority: normal Keywords: patch

Created on 2018-07-20 15:22 by vstinner, last changed 2018-08-23 10:44 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8353 merged vstinner, 2018-07-20 15:35
PR 8361 merged vstinner, 2018-07-21 01:28
PR 8417 merged vstinner, 2018-07-23 16:25
PR 8447 merged vstinner, 2018-07-24 18:53
PR 8454 merged vstinner, 2018-07-25 00:01
PR 8455 merged vstinner, 2018-07-25 00:51
PR 8468 merged vstinner, 2018-07-25 17:27
PR 8476 merged vstinner, 2018-07-26 10:14
PR 8484 merged vstinner, 2018-07-26 16:04
PR 8521 closed ronaldoussoren, 2018-07-28 12:36
PR 8591 merged vstinner, 2018-07-31 23:18
PR 8593 merged vstinner, 2018-08-01 00:08
PR 8594 merged vstinner, 2018-08-01 00:21
PR 8595 merged vstinner, 2018-08-01 00:27
PR 8607 merged vstinner, 2018-08-01 14:56
PR 8608 merged vstinner, 2018-08-01 16:04
PR 8631 merged vstinner, 2018-08-02 17:01
PR 8658 merged vstinner, 2018-08-03 20:19
PR 8660 merged vstinner, 2018-08-03 20:45
PR 8867 closed vstinner, 2018-08-23 09:51
PR 8868 merged vstinner, 2018-08-23 10:06
PR 8870 merged vstinner, 2018-08-23 10:27
Messages (27)
msg322015 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-20 15:24
Attached PR reworks _PyCoreConfig_Read() to leave _Py_path_config unchanged: *reading* the core configuration must not *modify* the path configuration.

The PR adds _PyCoreConfig.dll_path field.

The change is related to the PEP 432.
msg322018 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-20 15:27
See also bpo-34008: "Do we support calling Py_Main() after Py_Initialize()?".
msg322020 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-20 15:37
PR 8353 is the last piece of my long work on Py_Main() to separate code *reading* configuration and code *applying* a new configuration. My work on the new _PyPathConfig structure, and the PR should finish that work.

Extract of the commit message:

"A new _PyCoreConfig_SetPathConfig() function now recreates the path configuration from the core configuration. This function is now called very late in _Py_InitializeCore(), just before calling initimport()."
msg322040 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-21 00:06
New changeset b1147e43daeb3c51a63056b489e8d868404d4e22 by Victor Stinner in branch 'master':
bpo-34170: Rework _PyCoreConfig_Read() to avoid side effect (GH-8353)
https://github.com/python/cpython/commit/b1147e43daeb3c51a63056b489e8d868404d4e22
msg322060 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-21 01:54
New changeset f2626ce6d4136f13a506e34ca8631ff2eab85fd9 by Victor Stinner in branch 'master':
bpo-34170: _PyCoreConfig_Read() leaves Py_IsolatedFlag unchanged (GH-8361)
https://github.com/python/cpython/commit/f2626ce6d4136f13a506e34ca8631ff2eab85fd9
msg322290 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-24 11:55
New changeset d19d8d5279f156bc8f6736b5f16f069879b9519b by Victor Stinner in branch 'master':
bpo-34170: Add _PyCoreConfig.isolated (GH-8417)
https://github.com/python/cpython/commit/d19d8d5279f156bc8f6736b5f16f069879b9519b
msg322297 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-24 12:50
Reminder for myself: I have to write unit tests :-)
https://github.com/python/cpython/pull/8417#pullrequestreview-139835391
msg322336 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-25 00:49
New changeset 1dc6e3906acb81163725e98378bf4d1bd1ce771a by Victor Stinner in branch 'master':
bpo-34170: Add _Py_InitializeFromConfig() (GH-8454)
https://github.com/python/cpython/commit/1dc6e3906acb81163725e98378bf4d1bd1ce771a
msg322337 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-25 00:58
I added many fields to _PyCoreConfig which duplicate global configuration varibles: _PyConfigCore.isolated duplicates Py_IsolatedFlag.

I started to modify Python to read the core configuration rather than global configuration flags. The problem is that sometimes, global configuration flags are updated, but not their duplicated core configuration fields.

Example from Modules/main.c:

static void
pymain_repl(_PyMain *pymain, _PyCoreConfig *config, PyCompilerFlags *cf)
{
    /* Check this environment variable at the end, to give programs the
       opportunity to set it from Python. */
    if (!Py_InspectFlag && config_get_env_var(config, "PYTHONINSPECT")) {
        Py_InspectFlag = 1;
    }
    ...
}

Only Py_InspectFlag is not, not core_config.inspect.
msg322340 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-25 08:21
New changeset d3b191992e0f3c1eb27cb68478b875d60e859976 by Victor Stinner in branch 'master':
bpo-34170: Cleanup pymain_run_python() (GH-8455)
https://github.com/python/cpython/commit/d3b191992e0f3c1eb27cb68478b875d60e859976
msg322366 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-25 15:35
TODO list, open questions:

* Should isolated=1 always imply config->ignore_environment=1 and config->user_site_directory = 0?
* Should we rename debug to parser_debug? "debug" is a very generic term, whereas this flag is really specific to the parser
* Remove _Py_CheckHashBasedPycsMode: replace it with config._check_hash_pycs_mode?
msg322371 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-25 17:04
I created bpo-34228 "Allow PYTHONTRACEMALLOC=0 and -X tracemalloc=0 to disable explicitly tracemalloc" that I need to write unit tests on Python initiallization.
msg322393 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-26 00:37
New changeset ecf411c59e33d3760dbfebf6d5b4b205bcc29d5a by Victor Stinner in branch 'master':
bpo-34170: Enhance _PyCoreConfig_Read() (GH-8468)
https://github.com/python/cpython/commit/ecf411c59e33d3760dbfebf6d5b4b205bcc29d5a
msg322427 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-26 14:04
New changeset d145775b451bbae549391eb49ed612fb3b03bde1 by Victor Stinner in branch 'master':
bpo-34170: Cleanup pymain_read_conf() (GH-8476)
https://github.com/python/cpython/commit/d145775b451bbae549391eb49ed612fb3b03bde1
msg322440 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-26 16:58
New changeset 56b29b6d6fa3eb32bb1533ee3f21b1e7135648a0 by Victor Stinner in branch 'master':
bpo-34170, test_embed: write Py_Initialize() tests (GH-8484)
https://github.com/python/cpython/commit/56b29b6d6fa3eb32bb1533ee3f21b1e7135648a0
msg322828 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-01 00:13
New changeset b75d7e243512afcfb2285e6471262478383e09db by Victor Stinner in branch 'master':
bpo-34170: Add _PyCoreConfig._frozen parameter (GH-8591)
https://github.com/python/cpython/commit/b75d7e243512afcfb2285e6471262478383e09db
msg322829 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-01 00:57
New changeset a4d20b2e5ece2120f129cb4dda951a6c2461e92d by Victor Stinner in branch 'master':
bpo-34170: Py_Main() updates config when setting Py_InspectFlag (GH-8593)
https://github.com/python/cpython/commit/a4d20b2e5ece2120f129cb4dda951a6c2461e92d
msg322830 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-01 01:07
New changeset 9851227382431a40a138fdff994278d9e7743c74 by Victor Stinner in branch 'master':
bpo-34170: Rename _PyCoreConfig.unbuffered_stdip (GH-8594)
https://github.com/python/cpython/commit/9851227382431a40a138fdff994278d9e7743c74
msg322831 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-01 01:07
New changeset ea68d83933e6de6cabfb115ec1b8888301947369 by Victor Stinner in branch 'master':
bpo-34170: _PyCoreConfig_Read() defaults to argc=0 (GH-8595)
https://github.com/python/cpython/commit/ea68d83933e6de6cabfb115ec1b8888301947369
msg322868 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-01 15:56
New changeset 6c785c0ebdadc84d80a53d896c38fd7ada8ae1f6 by Victor Stinner in branch 'master':
bpo-34170: Add Python/coreconfig.c for _PyCoreConfig (GH-8607)
https://github.com/python/cpython/commit/6c785c0ebdadc84d80a53d896c38fd7ada8ae1f6
msg322870 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-01 16:18
New changeset 80b762f010097ab8137782e5fbdc89c5c620ed4e by Victor Stinner in branch 'master':
bpo-31650: Remove _Py_CheckHashBasedPycsMode global config var (GH-8608)
https://github.com/python/cpython/commit/80b762f010097ab8137782e5fbdc89c5c620ed4e
msg322871 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-01 16:18
"""
TODO list, open questions:

* Should isolated=1 always imply config->ignore_environment=1 and config->user_site_directory = 0?
* Should we rename debug to parser_debug? "debug" is a very generic term, whereas this flag is really specific to the parser
* Remove _Py_CheckHashBasedPycsMode: replace it with config._check_hash_pycs_mode?
"""

=> DONE!
msg322982 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-02 17:34
New changeset 72ec3193b5118a2ccc8be8bf03d7b74691c6a264 by Victor Stinner in branch 'master':
bpo-34170: Cleanup pymain_run_filename() (GH-8631)
https://github.com/python/cpython/commit/72ec3193b5118a2ccc8be8bf03d7b74691c6a264
msg323073 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-03 20:49
New changeset 5a953fd0ab4d0f792f4c1537616b2ce8ee40d976 by Victor Stinner in branch 'master':
bpo-34170: _PyCoreConfig_Read() don't replace coerce_c_locale (GH-8658)
https://github.com/python/cpython/commit/5a953fd0ab4d0f792f4c1537616b2ce8ee40d976
msg323086 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-03 21:54
New changeset d8078626770a8d358eb83d7928c12d75ff4e821a by Victor Stinner in branch 'master':
bpo-34170: Fix pymain_run_file() (GH-8660)
https://github.com/python/cpython/commit/d8078626770a8d358eb83d7928c12d75ff4e821a
msg323936 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 10:23
New changeset 89487f51b8d6ba8a55f5de0ed689e46fefe73cc9 by Victor Stinner in branch 'master':
bpo-34207: Fix pymain_read_conf() for UTF-8 Mode (GH-8868)
https://github.com/python/cpython/commit/89487f51b8d6ba8a55f5de0ed689e46fefe73cc9
msg323938 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 10:41
New changeset 80a0ebaa8385988b1d2dbbbb71ed1f548fc32ad7 by Victor Stinner in branch '3.7':
bpo-34207: Fix pymain_read_conf() for UTF-8 Mode (GH-8868) (GH-8870)
https://github.com/python/cpython/commit/80a0ebaa8385988b1d2dbbbb71ed1f548fc32ad7
History
Date User Action Args
2018-08-23 10:44:58vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-08-23 10:41:39vstinnersetmessages: + msg323938
2018-08-23 10:27:34vstinnersetpull_requests: + pull_request8347
2018-08-23 10:23:48vstinnersetmessages: + msg323936
2018-08-23 10:06:50vstinnersetpull_requests: + pull_request8344
2018-08-23 09:51:12vstinnersetpull_requests: + pull_request8342
2018-08-03 21:54:08vstinnersetmessages: + msg323086
2018-08-03 20:49:10vstinnersetmessages: + msg323073
2018-08-03 20:45:08vstinnersetpull_requests: + pull_request8154
2018-08-03 20:19:14vstinnersetpull_requests: + pull_request8152
2018-08-02 17:34:23vstinnersetmessages: + msg322982
2018-08-02 17:01:18vstinnersetpull_requests: + pull_request8136
2018-08-01 16:18:51vstinnersetmessages: + msg322871
2018-08-01 16:18:10vstinnersetmessages: + msg322870
2018-08-01 16:04:12vstinnersetpull_requests: + pull_request8114
2018-08-01 15:56:19vstinnersetmessages: + msg322868
2018-08-01 14:56:44vstinnersetpull_requests: + pull_request8112
2018-08-01 01:07:20vstinnersetmessages: + msg322831
2018-08-01 01:07:09vstinnersetmessages: + msg322830
2018-08-01 00:57:47vstinnersetmessages: + msg322829
2018-08-01 00:27:36vstinnersetpull_requests: + pull_request8105
2018-08-01 00:21:38vstinnersetpull_requests: + pull_request8104
2018-08-01 00:13:07vstinnersetmessages: + msg322828
2018-08-01 00:08:59vstinnersetpull_requests: + pull_request8103
2018-07-31 23:18:56vstinnersetpull_requests: + pull_request8101
2018-07-28 12:36:12ronaldoussorensetpull_requests: + pull_request8038
2018-07-26 16:58:02vstinnersetmessages: + msg322440
2018-07-26 16:04:35vstinnersetpull_requests: + pull_request8006
2018-07-26 14:04:58vstinnersetmessages: + msg322427
2018-07-26 10:14:29vstinnersetpull_requests: + pull_request8000
2018-07-26 00:37:28vstinnersetmessages: + msg322393
2018-07-25 17:27:20vstinnersetpull_requests: + pull_request7992
2018-07-25 17:04:13vstinnersetmessages: + msg322371
2018-07-25 15:35:59vstinnersetmessages: + msg322366
2018-07-25 08:21:10vstinnersetmessages: + msg322340
2018-07-25 00:58:51vstinnersetmessages: + msg322337
2018-07-25 00:51:59vstinnersetpull_requests: + pull_request7980
2018-07-25 00:49:20vstinnersetmessages: + msg322336
2018-07-25 00:01:18vstinnersetpull_requests: + pull_request7979
2018-07-24 18:53:14vstinnersetpull_requests: + pull_request7969
2018-07-24 12:50:24vstinnersetmessages: + msg322297
2018-07-24 11:55:54vstinnersetmessages: + msg322290
2018-07-23 16:25:50vstinnersetpull_requests: + pull_request7943
2018-07-21 01:54:22vstinnersetmessages: + msg322060
2018-07-21 01:28:37vstinnersetpull_requests: + pull_request7896
2018-07-21 00:06:20vstinnersetmessages: + msg322040
2018-07-20 15:37:25vstinnersetmessages: + msg322020
2018-07-20 15:35:13vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request7888
2018-07-20 15:27:25vstinnersetmessages: + msg322018
2018-07-20 15:26:38vstinnersetmessages: - msg322014
2018-07-20 15:24:12vstinnersetmessages: + msg322015
2018-07-20 15:22:57vstinnercreate