Author vstinner
Recipients ncoghlan, vstinner
Date 2019-03-06.00:45:14
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
Description of this long serie of changes.

I modified Py_Main() and _Py_InitializeCore() to clearly separate "pre-configuration" from "configuration" steps. The pre-configuration now decodes temporarily the command line arguments and uses its own command line parser to get -E, -I and -X options (-X is needed for -X utf8). The pre-configuration is designed to be as small as possible, it configures:

* memory allocators
* LC_CTYPE locale and set the UTF-8 mode

The _PyPreConfig structure has 8 fields:

* allocator
* coerce_c_locale
* coerce_c_locale_warn
* dev_mode
* isolated
* (Windows only) legacy_windows_fs_encoding
* use_environment
* utf8_mode

I had to include fields which have an impact on other fields. Examples:

* dev_mode=1 sets allocator to "default";
* isolated=1 sets use_environment to 0;
* legacy_windows_fs_encoding=& sets utf8_mode to 0.

_PyCoreConfig_Read() is now only called after the memory allocator and the locale (LC_CTYPE locale and UTF-8 mode) are properly configured.

I removed the last side effects of _PyCoreConfig_Read(): it no longer modify the locale. Same for the new _PyPreConfig_Read(): zero size effect.

The new _PyPreConfig_Write() and _PyCoreConfig_Write() are now responsible to write the new configurations.

There are functions to read the configuration from command line arguments:

* _PyPreConfig_ReadFromArgv()
* _PyCoreConfig_ReadFromArgv()

These functions expect a _PyArgv structure which accepts bytes (wchar*) or Unicode (wchar_t*).

I moved coreconfig.h from Include/ to Include/cpython/ to be more explicit that it's excluded from the stable API and that it's CPython specific.

I moved all config functions to a new Include/internal/pycore_coreconfig.h. Functions are internal to allow us to modify us anytime until a proper clean public API is designed on top of it.

If _PyPreConfig.allocator is set, _PyPreConfig_Write() re-allocate the configuration with the new memory allocator. This tiny thing avoids the new to force a specific memory allocator in many functions. I was able to remove the following code:

    PyMemAllocatorEx old_alloc;
    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
    PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

Calling Py_Main() after Py_Initialize() is still supported. In this case, it no longer checks the memory allocators name because _PyMem_GetAllocatorsName() returns "pymalloc_debug" (or "malloc_debug" if pymalloc is disabled) after _PyMem_SetupAllocators("debug") is called: names are diffrent.
Date User Action Args
2019-03-06 00:45:15vstinnersetrecipients: + vstinner, ncoghlan
2019-03-06 00:45:15vstinnersetmessageid: <>
2019-03-06 00:45:15vstinnerlinkissue36142 messages
2019-03-06 00:45:14vstinnercreate