Author vstinner
Recipients methane, ncoghlan, vstinner
Date 2019-03-06.01:22:29
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1551835349.16.0.0652835322584.issue36202@roundup.psfhosted.org>
In-reply-to
Content
The "vim" editor embeds Python. It sets the Python home by calling Py_SetPythonHome() with the following code:
---
	    size_t len = mbstowcs(NULL, (char *)p_py3home, 0) + 1;

	    /* The string must not change later, make a copy in static memory. */
	    py_home_buf = (wchar_t *)alloc(len * sizeof(wchar_t));
	    if (py_home_buf != NULL && mbstowcs(
			    py_home_buf, (char *)p_py3home, len) != (size_t)-1)
		Py_SetPythonHome(py_home_buf);
---
ref: https://github.com/vim/vim/blob/14816ad6e58336773443f5ee2e4aa9e384af65d2/src/if_python3.c#L874-L887

mbstowcs() uses the current LC_CTYPE locale. Python can select a different filesystem encoding than the LC_CTYPE encoding depending on PEP 538 and PEP 540. So encoding back the Python home to bytes to access to files on the filesystem can fail because of mojibake.

The code should by written like (pseudo-code):
---
_Py_PreInitialize();

_PyCoreConfig config;
config.home = Py_DecodeLocale(p_py3home);
if (config.home == NULL) { /* ERROR */ }

_PyInitError err = _Py_InitializeFromConfig(&config);
if (_Py_INIT_FAILED(err)) {
    _PyCoreConfig_Clear(&config);
    _Py_ExitInitError(err);
}
---

The vim case has been discussed at:
https://discuss.python.org/t/adding-char-based-apis-for-unix/916/8
History
Date User Action Args
2019-03-06 01:22:29vstinnersetrecipients: + vstinner, ncoghlan, methane
2019-03-06 01:22:29vstinnersetmessageid: <1551835349.16.0.0652835322584.issue36202@roundup.psfhosted.org>
2019-03-06 01:22:29vstinnerlinkissue36202 messages
2019-03-06 01:22:29vstinnercreate