classification
Title: Add _Py_PreInitialize() function
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: vstinner
Priority: normal Keywords: patch

Created on 2019-03-15 12:29 by vstinner, last changed 2019-04-05 09:44 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12343 merged vstinner, 2019-03-15 13:12
PR 12347 merged vstinner, 2019-03-15 14:11
PR 12350 closed vstinner, 2019-03-15 15:18
PR 12351 closed vstinner, 2019-03-15 15:34
PR 12420 merged vstinner, 2019-03-18 21:00
PR 12422 merged vstinner, 2019-03-19 00:03
PR 12457 merged vstinner, 2019-03-20 00:56
PR 12458 merged vstinner, 2019-03-20 01:48
PR 12506 merged vstinner, 2019-03-23 02:41
PR 12535 merged vstinner, 2019-03-25 13:46
PR 12536 merged vstinner, 2019-03-25 17:01
PR 12540 merged vstinner, 2019-03-25 21:31
PR 12542 merged vstinner, 2019-03-25 22:23
PR 12546 merged vstinner, 2019-03-25 23:49
PR 12563 merged vstinner, 2019-03-26 15:19
PR 12569 merged vstinner, 2019-03-26 22:53
PR 12695 merged vstinner, 2019-04-05 08:52
Messages (20)
msg337982 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-15 12:29
Follow-up of bpo-36142, add _Py_PreInitialize() function to "pre-initialize" Python:

* initialize memory allocators
* initialize LC_CTYPE locale and UTF-8 Mode

Py_Initialize() should also be modified to no longer coerce the C locale or enable the UTF-8 Mode:
https://bugs.python.org/issue36202#msg337915

See also:

* bpo-36202: Calling Py_DecodeLocale() before _PyPreConfig_Write() can produce mojibake
* bpo-36204: Deprecate calling Py_Main() after Py_Initialize()? Add Py_InitializeFromArgv()?
msg337991 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-15 14:08
New changeset 74f6568bbd3e70806ea3219e8bacb386ad802ccf by Victor Stinner in branch 'master':
bpo-36301: Add _PyWstrList structure (GH-12343)
https://github.com/python/cpython/commit/74f6568bbd3e70806ea3219e8bacb386ad802ccf
msg337993 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-15 15:03
New changeset 625997622b4736e9184bdd8bf1e22a7b51be1afc by Victor Stinner in branch 'master':
bpo-36301: _PyCoreConfig_Read() ensures that argv is not empty (GH-12347)
https://github.com/python/cpython/commit/625997622b4736e9184bdd8bf1e22a7b51be1afc
msg338287 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-18 21:24
New changeset c183444f7e2640b054956474d71aae6e8d31a543 by Victor Stinner in branch 'master':
bpo-36301: Fix Py_Main() memory leaks (GH-12420)
https://github.com/python/cpython/commit/c183444f7e2640b054956474d71aae6e8d31a543
msg338303 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-19 00:46
New changeset 5f9cf23502febe0eb3bc02e45c7d2bfc79424757 by Victor Stinner in branch 'master':
bpo-36301: Error if decoding pybuilddir.txt fails (GH-12422)
https://github.com/python/cpython/commit/5f9cf23502febe0eb3bc02e45c7d2bfc79424757
msg338428 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-20 01:20
New changeset f29084d611a6ca504c99a0967371374febf0ccc3 by Victor Stinner in branch 'master':
bpo-36301: Add _PyRuntime.pre_initialized (GH-12457)
https://github.com/python/cpython/commit/f29084d611a6ca504c99a0967371374febf0ccc3
msg338440 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-20 03:25
New changeset fa1537684869186da7938e4330361bf02363bac8 by Victor Stinner in branch 'master':
bpo-36301: Add _PyPreCmdline internal API (GH-12458)
https://github.com/python/cpython/commit/fa1537684869186da7938e4330361bf02363bac8
msg338484 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-20 15:32
In term of API, we get something like:

    _PyInitError err;

    _PyPreConfig preconfig = _PyPreConfig_INIT;
    preconfig.utf8_mode = 1;
    preconfig.allocator = "malloc";

    _PyInitError err = _Py_PreInitializeFromPreConfig(&preconfig);
    if (_Py_INIT_FAILED(err)) {
        _Py_ExitInitError(err);
    }

    /* PyMem_RawMalloc and Py_DecodeLocale can now be used */

    _PyCoreConfig config = _PyCoreConfig_INIT;
    config.user_site_directory = 0;

    err = _Py_InitializeFromConfig(&config);
    if (_Py_INIT_FAILED(err)) {
        _Py_ExitInitError(err);
    }

    /* ... use Python ... */

    Py_Finalize();

    /* Note: no need to "free" preconfig nor config memory, they use constants */
msg338664 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-23 11:06
New changeset 6d5ee973f0600a3a9444f569dcf0dd346bfa2a11 by Victor Stinner in branch 'master':
bpo-36301: Add _PyRuntimeState.preconfig (GH-12506)
https://github.com/python/cpython/commit/6d5ee973f0600a3a9444f569dcf0dd346bfa2a11
msg338809 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-25 16:55
New changeset f72346c47537657a287a862305f65eb5d7594fbf by Victor Stinner in branch 'master':
bpo-36301: Cleanup preconfig code (GH-12535)
https://github.com/python/cpython/commit/f72346c47537657a287a862305f65eb5d7594fbf
msg338810 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-25 16:58
Note for myself: is there a problem between the priority of PYTHONHOME env var and pybuilddir.txt configuration file?
msg338814 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-25 17:37
New changeset a6fbc4e25e1dc7d1c9a26888b9115bc6c2afc101 by Victor Stinner in branch 'master':
bpo-36301: Add _Py_PreInitializeFromConfig() (GH-12536)
https://github.com/python/cpython/commit/a6fbc4e25e1dc7d1c9a26888b9115bc6c2afc101
msg338829 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-25 22:19
New changeset 1075d1684ab84dc7c28d93cfb46e95e70d3b6d3b by Victor Stinner in branch 'master':
bpo-36301: Add _Py_GetConfigsAsDict() function (GH-12540)
https://github.com/python/cpython/commit/1075d1684ab84dc7c28d93cfb46e95e70d3b6d3b
msg338837 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-25 23:03
New changeset f78a5e9ce8f32a195f5f788aade79578437f30a6 by Victor Stinner in branch 'master':
bpo-36301: Add _Py_GetEnv() function (GH-12542)
https://github.com/python/cpython/commit/f78a5e9ce8f32a195f5f788aade79578437f30a6
msg338846 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-26 01:31
New changeset 20004959d23d07ac784eef51ecb161012180faa8 by Victor Stinner in branch 'master':
bpo-36301: Remove _PyCoreConfig.preconfig (GH-12546)
https://github.com/python/cpython/commit/20004959d23d07ac784eef51ecb161012180faa8
msg338895 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-26 15:24
Note for myself: PYTHONDEVMODE=1, PreConfig isolated=1, CoreConfig isolated=0: is the dev mode enabled or not? IMHO it should not. Maybe add a specific unit test?
msg338903 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-26 15:59
New changeset f8ba6f5afc317d1be3025db1be410ac66a7e5a27 by Victor Stinner in branch 'master':
bpo-36301: Cleanup preconfig.c and coreconfig.c (GH-12563)
https://github.com/python/cpython/commit/f8ba6f5afc317d1be3025db1be410ac66a7e5a27
msg338923 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-26 23:05
I created a follow-up issue: bpo-36443.

--

> Note for myself: PYTHONDEVMODE=1, PreConfig isolated=1, CoreConfig isolated=0: is the dev mode enabled or not? IMHO it should not. Maybe add a specific unit test?

PR 12569 adds these tests.
msg338925 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-26 23:55
The feature has been implemented, I close the issue. The work is continued in other issues.
msg339483 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-05 09:44
New changeset 6a8c3139ae9ada89d4a95985ec7cf8bb7d03bc01 by Victor Stinner in branch 'master':
bpo-36301: Fix _PyPreConfig_Read() compiler warning (GH-12695)
https://github.com/python/cpython/commit/6a8c3139ae9ada89d4a95985ec7cf8bb7d03bc01
History
Date User Action Args
2019-04-05 09:44:09vstinnersetmessages: + msg339483
2019-04-05 08:52:44vstinnersetpull_requests: + pull_request12620
2019-03-26 23:55:16vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg338925

stage: patch review -> resolved
2019-03-26 23:05:26vstinnersetmessages: + msg338923
2019-03-26 22:53:08vstinnersetpull_requests: + pull_request12514
2019-03-26 15:59:04vstinnersetmessages: + msg338903
2019-03-26 15:24:47vstinnersetmessages: + msg338895
2019-03-26 15:19:31vstinnersetpull_requests: + pull_request12508
2019-03-26 01:31:15vstinnersetmessages: + msg338846
2019-03-25 23:49:37vstinnersetpull_requests: + pull_request12495
2019-03-25 23:03:17vstinnersetmessages: + msg338837
2019-03-25 22:23:10vstinnersetpull_requests: + pull_request12491
2019-03-25 22:19:59vstinnersetmessages: + msg338829
2019-03-25 21:31:46vstinnersetpull_requests: + pull_request12489
2019-03-25 17:37:15vstinnersetmessages: + msg338814
2019-03-25 17:01:41vstinnersetpull_requests: + pull_request12486
2019-03-25 16:58:34vstinnersetmessages: + msg338810
2019-03-25 16:55:09vstinnersetmessages: + msg338809
2019-03-25 13:46:13vstinnersetpull_requests: + pull_request12485
2019-03-23 11:06:03vstinnersetmessages: + msg338664
2019-03-23 02:41:40vstinnersetpull_requests: + pull_request12457
2019-03-20 15:32:41vstinnersetmessages: + msg338484
2019-03-20 03:25:40vstinnersetmessages: + msg338440
2019-03-20 01:48:13vstinnersetpull_requests: + pull_request12411
2019-03-20 01:20:15vstinnersetmessages: + msg338428
2019-03-20 00:56:47vstinnersetpull_requests: + pull_request12410
2019-03-19 00:46:27vstinnersetmessages: + msg338303
2019-03-19 00:03:12vstinnersetpull_requests: + pull_request12377
2019-03-18 21:24:30vstinnersetmessages: + msg338287
2019-03-18 21:00:37vstinnersetpull_requests: + pull_request12374
2019-03-15 15:34:25vstinnersetpull_requests: + pull_request12317
2019-03-15 15:18:08vstinnersetpull_requests: + pull_request12316
2019-03-15 15:03:28vstinnersetmessages: + msg337993
2019-03-15 14:11:58vstinnersetpull_requests: + pull_request12313
2019-03-15 14:08:09vstinnersetmessages: + msg337991
2019-03-15 13:12:20vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request12310
2019-03-15 12:29:28vstinnercreate