Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add _Py_PreInitialize() function #80482

Closed
vstinner opened this issue Mar 15, 2019 · 20 comments
Closed

Add _Py_PreInitialize() function #80482

vstinner opened this issue Mar 15, 2019 · 20 comments
Labels
3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@vstinner
Copy link
Member

BPO 36301
Nosy @vstinner
PRs
  • bpo-36301: Add _PyWstrList structure #12343
  • bpo-36301: _PyCoreConfig_Read() ensures that argv is non-empty #12347
  • bpo-36301: Add _PyRuntimeState.preconfig #12350
  • [WIP] bpo-36301: Add _Py_PreInitialize() function #12351
  • bpo-36301: Fix Py_Main() memory leaks #12420
  • bpo-36301: Error if decoding pybuilddir.txt fails #12422
  • bpo-36301: Add _PyRuntime.pre_initialized #12457
  • bpo-36301: Add _PyPreCmdline internal API #12458
  • bpo-36301: Add _PyRuntimeState.preconfig #12506
  • bpo-36301: Cleanup preconfig code #12535
  • bpo-36301: Add _Py_PreInitializeFromConfig() #12536
  • bpo-36301: test_embed separates pre_config #12540
  • bpo-36301: Add _Py_GetEnv() function #12542
  • bpo-36301: Remove _PyCoreConfig.preconfig #12546
  • bpo-36301: Cleanup preconfig.c and coreconfig.c #12563
  • bpo-36301: Test Python init with isolated #12569
  • bpo-36301: Fix _PyPreConfig_Read() compiler warning #12695
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2019-03-26.23:55:16.532>
created_at = <Date 2019-03-15.12:29:28.556>
labels = ['interpreter-core', '3.8']
title = 'Add _Py_PreInitialize() function'
updated_at = <Date 2019-04-05.09:44:09.603>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2019-04-05.09:44:09.603>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-03-26.23:55:16.532>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2019-03-15.12:29:28.556>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 36301
keywords = ['patch']
message_count = 20.0
messages = ['337982', '337991', '337993', '338287', '338303', '338428', '338440', '338484', '338664', '338809', '338810', '338814', '338829', '338837', '338846', '338895', '338903', '338923', '338925', '339483']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['12343', '12347', '12350', '12351', '12420', '12422', '12457', '12458', '12506', '12535', '12536', '12540', '12542', '12546', '12563', '12569', '12695']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36301'
versions = ['Python 3.8']
    @vstinner
    Copy link
    Member Author

    Follow-up of bpo-36142, add _Py_PreInitialize() function to "pre-initialize" Python:

    • initialize memory allocators
    • initialize LC_CTYPE locale and UTF-8 Mode

    Py_Initialize() should also be modified to no longer coerce the C locale or enable the UTF-8 Mode:
    https://bugs.python.org/issue36202#msg337915

    See also:

    • bpo-36202: Calling Py_DecodeLocale() before _PyPreConfig_Write() can produce mojibake
    • bpo-36204: Deprecate calling Py_Main() after Py_Initialize()? Add Py_InitializeFromArgv()?

    @vstinner vstinner added 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Mar 15, 2019
    @vstinner
    Copy link
    Member Author

    New changeset 74f6568 by Victor Stinner in branch 'master':
    bpo-36301: Add _PyWstrList structure (GH-12343)
    74f6568

    @vstinner
    Copy link
    Member Author

    New changeset 6259976 by Victor Stinner in branch 'master':
    bpo-36301: _PyCoreConfig_Read() ensures that argv is not empty (GH-12347)
    6259976

    @vstinner
    Copy link
    Member Author

    New changeset c183444 by Victor Stinner in branch 'master':
    bpo-36301: Fix Py_Main() memory leaks (GH-12420)
    c183444

    @vstinner
    Copy link
    Member Author

    New changeset 5f9cf23 by Victor Stinner in branch 'master':
    bpo-36301: Error if decoding pybuilddir.txt fails (GH-12422)
    5f9cf23

    @vstinner
    Copy link
    Member Author

    New changeset f29084d by Victor Stinner in branch 'master':
    bpo-36301: Add _PyRuntime.pre_initialized (GH-12457)
    f29084d

    @vstinner
    Copy link
    Member Author

    New changeset fa15376 by Victor Stinner in branch 'master':
    bpo-36301: Add _PyPreCmdline internal API (GH-12458)
    fa15376

    @vstinner
    Copy link
    Member Author

    In term of API, we get something like:

    _PyInitError err;

_PyPreConfig preconfig = _PyPreConfig_INIT;
preconfig.utf8_mode = 1;
preconfig.allocator = "malloc";

        _PyInitError err = _Py_PreInitializeFromPreConfig(&preconfig);
    if (_Py_INIT_FAILED(err)) {
        _Py_ExitInitError(err);
    }
    /* PyMem_RawMalloc and Py_DecodeLocale can now be used */

_PyCoreConfig config = _PyCoreConfig_INIT;
config.user_site_directory = 0;

        err = _Py_InitializeFromConfig(&config);
    if (_Py_INIT_FAILED(err)) {
        _Py_ExitInitError(err);
    }
    /* ... use Python ... */

        Py_Finalize();
    /* Note: no need to "free" preconfig nor config memory, they use constants */

    @vstinner
    Copy link
    Member Author

    New changeset 6d5ee97 by Victor Stinner in branch 'master':
    bpo-36301: Add _PyRuntimeState.preconfig (GH-12506)
    6d5ee97

    @vstinner
    Copy link
    Member Author

    New changeset f72346c by Victor Stinner in branch 'master':
    bpo-36301: Cleanup preconfig code (GH-12535)
    f72346c

    @vstinner
    Copy link
    Member Author

    Note for myself: is there a problem between the priority of PYTHONHOME env var and pybuilddir.txt configuration file?

    @vstinner
    Copy link
    Member Author

    New changeset a6fbc4e by Victor Stinner in branch 'master':
    bpo-36301: Add _Py_PreInitializeFromConfig() (GH-12536)
    a6fbc4e

    @vstinner
    Copy link
    Member Author

    New changeset 1075d16 by Victor Stinner in branch 'master':
    bpo-36301: Add _Py_GetConfigsAsDict() function (GH-12540)
    1075d16

    @vstinner
    Copy link
    Member Author

    New changeset f78a5e9 by Victor Stinner in branch 'master':
    bpo-36301: Add _Py_GetEnv() function (GH-12542)
    f78a5e9

    @vstinner
    Copy link
    Member Author

    New changeset 2000495 by Victor Stinner in branch 'master':
    bpo-36301: Remove _PyCoreConfig.preconfig (GH-12546)
    2000495

    @vstinner
    Copy link
    Member Author

    Note for myself: PYTHONDEVMODE=1, PreConfig isolated=1, CoreConfig isolated=0: is the dev mode enabled or not? IMHO it should not. Maybe add a specific unit test?

    @vstinner
    Copy link
    Member Author

    New changeset f8ba6f5 by Victor Stinner in branch 'master':
    bpo-36301: Cleanup preconfig.c and coreconfig.c (GH-12563)
    f8ba6f5

    @vstinner
    Copy link
    Member Author

    I created a follow-up issue: bpo-36443.

    --

    Note for myself: PYTHONDEVMODE=1, PreConfig isolated=1, CoreConfig isolated=0: is the dev mode enabled or not? IMHO it should not. Maybe add a specific unit test?

    PR 12569 adds these tests.

    @vstinner
    Copy link
    Member Author

    The feature has been implemented, I close the issue. The work is continued in other issues.

    @vstinner
    Copy link
    Member Author

    vstinner commented Apr 5, 2019

    New changeset 6a8c313 by Victor Stinner in branch 'master':
    bpo-36301: Fix _PyPreConfig_Read() compiler warning (GH-12695)
    6a8c313

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @vstinner