classification
Title: The value of Py_SetPath is not used to populate the configuration
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: koobs, lukasz.langa, miss-islington, pablogsal, steve.dower, vstinner
Priority: Keywords: 3.8regression, patch

Created on 2019-09-20 15:19 by pablogsal, last changed 2019-09-27 20:05 by vstinner.

Pull Requests
URL Status Linked Edit
PR 16298 merged vstinner, 2019-09-20 15:41
PR 16307 merged vstinner, 2019-09-20 22:39
PR 16312 closed miss-islington, 2019-09-20 23:50
PR 16313 merged vstinner, 2019-09-20 23:53
PR 16335 merged vstinner, 2019-09-23 15:55
PR 16336 merged vstinner, 2019-09-23 17:04
PR 16338 merged vstinner, 2019-09-23 22:23
PR 16339 closed vstinner, 2019-09-23 22:27
PR 16340 merged miss-islington, 2019-09-23 22:56
PR 16357 merged vstinner, 2019-09-24 15:11
PR 16358 merged vstinner, 2019-09-24 16:01
PR 16366 merged vstinner, 2019-09-24 23:50
PR 16367 merged vstinner, 2019-09-25 00:32
PR 16390 merged vstinner, 2019-09-25 14:12
PR 16402 merged vstinner, 2019-09-25 22:00
PR 16404 merged vstinner, 2019-09-26 00:26
PR 16406 merged vstinner, 2019-09-26 01:40
PR 16421 merged vstinner, 2019-09-26 13:19
PR 16423 merged vstinner, 2019-09-26 14:05
Messages (32)
msg352856 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-20 15:19
When calling Py_SetPath(), the value that is passed in is ignored.
msg352857 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-20 15:20
It's a 3.8 regression: I set the priority to release blocker.
msg352859 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-20 15:24
A very simple way to reproduce the bug:

Using pyInstaller, after applying these patches (to fix other Python3.8 issues)

https://github.com/pyinstaller/pyinstaller/pull/4441
https://github.com/pyinstaller/pyinstaller/pull/4440

you will get

Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

This is because Python will use the default value for sys.path, including the "lib" subdirectory:

/dist/hello/lib/python3.8
            ^----------------> This should not be here

Instead, that value should be the one that pyinstaller passes when calling Py_SetPath() here: 

https://github.com/pyinstaller/pyinstaller/blob/develop/bootloader/src/pyi_pythonlib.c#L495

So it should be:

/dist/hello/python3.8
msg352862 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-20 15:43
That's a regression that I introduced between Python 3.7 and 3.8 when refactoring _PyCoreConfig/_PyPathConfig code into PyConfig/_PyPathConfig. 

Previously, if Py_SetPath() was called, the specified string was used to fill config->module_search_paths. Now, this string is basically ignored.

PR 16298 fix this regression.
msg352871 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-20 16:57
> Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding

I created bpo-38236 "Dump the Python path configuration at the first import error" which should ease to debug such issues.
msg352909 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-20 23:02
New changeset c422167749f92d4170203e996a2c619c818335ea by Victor Stinner in branch 'master':
bpo-38234: Remove _PyPathConfig.dll_path (GH-16307)
https://github.com/python/cpython/commit/c422167749f92d4170203e996a2c619c818335ea
msg352913 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-20 23:50
New changeset e267793aa4101b2771ed0e66aaff5743d23f59af by Victor Stinner in branch 'master':
bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298)
https://github.com/python/cpython/commit/e267793aa4101b2771ed0e66aaff5743d23f59af
msg352914 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-21 00:13
New changeset 9f3dcf802eefeb5ab821ce3c7204ab46557d53d7 by Victor Stinner in branch '3.8':
[3.8] bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298) (GH-16313)
https://github.com/python/cpython/commit/9f3dcf802eefeb5ab821ce3c7204ab46557d53d7
msg352915 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-21 00:13
> When calling Py_SetPath(), the value that is passed in is ignored.

I fixed this regression. Thanks for the bug report.
msg353030 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-09-23 15:09
Why are we keeping the DLL path around at all? It should only be being used in 1-2 places during path calculation.
msg353031 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-23 15:16
Steve:
> Why are we keeping the DLL path around at all? It should only be being used in 1-2 places during path calculation.

_PyPathConfig_Init() initializes _Py_dll_path global variable which is only used by _Py_CheckPython3(). _Py_CheckPython3() is called at each _PyImport_FindSharedFuncptrWindows() call which is used by _PyImport_LoadDynamicModuleWithSpec() (to import a .pyd extension). I tried to minimize my changes to fix the issue, I tried to leave _Py_CheckPython3() unchanged. But I made one change in _Py_CheckPython3(): it handles the case _Py_dll_path=NULL... which should never occur, but I wasn't 100% sure that it really can never occur.

Python 3.6 (before my init work), _Py_CheckPython3() used "static wchar_t dllpath[MAXPATHLEN+1];": it was less important when dllpath is initialized.

--

_PyPathConfig_Calculate() of PC/getpathp.c now uses a temporary variable which stores the result of _Py_GetDLLPath().
msg353032 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-09-23 15:39
> _PyPathConfig_Init() initializes _Py_dll_path global variable which is only used by _Py_CheckPython3().

Ah okay, maybe I'll take the time later on (in a separate issue) to get rid of it fully. We may as well load python3.dll at startup anyway, it's not expensive.
msg353034 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-23 16:12
> When calling Py_SetPath(), the value that is passed in is ignored.

Hum, in fact the bug was even worse: Py_SetPythonHome() and Py_SetProgramName() calls are also ignored. I'm not sure when I introduced these regressions.

PR 16335 fix this bug as well.
msg353035 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-23 16:47
New changeset 9c42f8cda552694f3b47d6388d4ae84d61731872 by Victor Stinner in branch 'master':
bpo-38234: Fix _PyConfig_InitPathConfig() (GH-16335)
https://github.com/python/cpython/commit/9c42f8cda552694f3b47d6388d4ae84d61731872
msg353037 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-23 17:50
New changeset 3f5409a3f13c59baa34656bccefdc3728e46c9ef by Victor Stinner in branch '3.8':
bpo-38234: Fix _PyConfig_InitPathConfig() (GH-16335) (GH-16336)
https://github.com/python/cpython/commit/3f5409a3f13c59baa34656bccefdc3728e46c9ef
msg353042 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-23 22:55
New changeset 85ce0a7178801b538160cbb5cf9ef50a713c45bf by Victor Stinner in branch 'master':
bpo-38234: read_pth_file() now returns PyStatus (GH-16338)
https://github.com/python/cpython/commit/85ce0a7178801b538160cbb5cf9ef50a713c45bf
msg353043 - (view) Author: miss-islington (miss-islington) Date: 2019-09-23 23:16
New changeset 7f7cd899e3d773b803b7af5b0c19eeff83dd69fe by Miss Islington (bot) in branch '3.8':
bpo-38234: read_pth_file() now returns PyStatus (GH-16338)
https://github.com/python/cpython/commit/7f7cd899e3d773b803b7af5b0c19eeff83dd69fe
msg353096 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-24 15:44
New changeset 1ce152a42eaa917d7763bce93f1e1ca72530d7ca by Victor Stinner in branch 'master':
bpo-38234: Py_SetPath() uses the program full path (GH-16357)
https://github.com/python/cpython/commit/1ce152a42eaa917d7763bce93f1e1ca72530d7ca
msg353099 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-24 16:21
New changeset bb6bf7d342b4503a6227fd209fac934905b6a1aa by Victor Stinner in branch 'master':
bpo-38234: Add tests for Python init path config (GH-16358)
https://github.com/python/cpython/commit/bb6bf7d342b4503a6227fd209fac934905b6a1aa
msg353124 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-25 00:10
New changeset 52ad33abbfb6637d74932617c7013bae0ccf6e32 by Victor Stinner in branch 'master':
bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt (GH-16366)
https://github.com/python/cpython/commit/52ad33abbfb6637d74932617c7013bae0ccf6e32
msg353128 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-25 00:54
New changeset 221fd84703c545408bbb4a6e0b58459651331f5c by Victor Stinner in branch 'master':
bpo-38234: Cleanup getpath.c (GH-16367)
https://github.com/python/cpython/commit/221fd84703c545408bbb4a6e0b58459651331f5c
msg353129 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-25 00:57
The new tests fail on macOS.

x86-64 High Sierra 3.x:
https://buildbot.python.org/all/#/builders/145/builds/2469

======================================================================
FAIL: test_init_pybuilddir (test.test_embed.InitConfigTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 1130, in test_init_pybuilddir
    self.check_all_configs("test_init_compat_config", config,
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[655 chars]ne, 'module_search_paths': ['/Users/buildbot/b[1407 chars]': 1} != {'_co[655 chars]ne, 'executable': '/var/folders/sy/9hwmqyx14s1[1383 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
-  'base_executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp2a98rb5k/_testembed',
?                      --------

+  'base_executable': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp2a98rb5k/_testembed',
   'base_prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
   'buffered_stdio': 1,
   'bytes_warning': 0,
   'check_hash_pycs_mode': 'default',
   'configure_c_stdio': 0,
   'dev_mode': 0,
   'dump_refs': 0,
   'exec_prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
-  'executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp2a98rb5k/_testembed',
?                 --------

+  'executable': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp2a98rb5k/_testembed',
   'faulthandler': 0,
   'hash_seed': 0,
   'home': None,
   'import_time': 0,
   'inspect': 0,
   'install_signal_handlers': 1,
   'interactive': 0,
   'isolated': 0,
   'malloc_stats': 0,
   'module_search_paths': ['/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python39.zip',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/build/lib.macosx-10.13-x86_64-3.9-pydebug',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python39.zip',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python3.9',
-                          '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp2a98rb5k/libdir'],
?                           --------

+                          '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp2a98rb5k/libdir'],
   'optimization_level': 0,
   'parse_argv': 0,
   'parser_debug': 0,
   'pathconfig_warnings': 1,
   'prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
   'program_name': './_testembed',
   'pycache_prefix': None,
   'pythonpath_env': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python39.zip:/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib:/Users/buildbot/buildarea/3.x.billenstein-sierra/build/build/lib.macosx-10.13-x86_64-3.9-pydebug',
   'quiet': 0,
   'run_command': None,
   'run_filename': None,
   'run_module': None,
   'show_alloc_count': 0,
   'show_ref_count': 0,
   'site_import': 1,
   'skip_source_first_line': 0,
   'tracemalloc': 0,
   'use_environment': 1,
   'use_hash_seed': 0,
   'user_site_directory': 1,
   'verbose': 0,
   'warnoptions': [],
   'write_bytecode': 1,
   'xoptions': []}

======================================================================
FAIL: test_init_pyvenv_cfg (test.test_embed.InitConfigTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 1182, in test_init_pyvenv_cfg
    self.check_all_configs("test_init_compat_config", config,
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[655 chars]ne, 'module_search_paths': ['/Users/buildbot/b[1416 chars]': 1} != {'_co[655 chars]ne, 'executable': '/var/folders/sy/9hwmqyx14s1[1400 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmprrz3csek',
-  'base_executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmpczgnz4r3/_testembed',
?                      --------

+  'base_executable': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmpczgnz4r3/_testembed',
   'base_prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
   'buffered_stdio': 1,
   'bytes_warning': 0,
   'check_hash_pycs_mode': 'default',
   'configure_c_stdio': 0,
   'dev_mode': 0,
   'dump_refs': 0,
   'exec_prefix': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmprrz3csek',
-  'executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmpczgnz4r3/_testembed',
?                 --------

+  'executable': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmpczgnz4r3/_testembed',
   'faulthandler': 0,
   'hash_seed': 0,
   'home': None,
   'import_time': 0,
   'inspect': 0,
   'install_signal_handlers': 1,
   'interactive': 0,
   'isolated': 0,
   'malloc_stats': 0,
   'module_search_paths': ['/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python39.zip',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/build/lib.macosx-10.13-x86_64-3.9-pydebug',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python39.zip',
                           '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python3.9',
                           '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmprrz3csek/lib/python3.9/lib-dynload'],
   'optimization_level': 0,
   'parse_argv': 0,
   'parser_debug': 0,
   'pathconfig_warnings': 1,
   'prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
   'program_name': './_testembed',
   'pycache_prefix': None,
   'pythonpath_env': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target/lib/python39.zip:/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib:/Users/buildbot/buildarea/3.x.billenstein-sierra/build/build/lib.macosx-10.13-x86_64-3.9-pydebug',
   'quiet': 0,
   'run_command': None,
   'run_filename': None,
   'run_module': None,
   'show_alloc_count': 0,
   'show_ref_count': 0,
   'site_import': 1,
   'skip_source_first_line': 0,
   'tracemalloc': 0,
   'use_environment': 1,
   'use_hash_seed': 0,
   'user_site_directory': 1,
   'verbose': 0,
   'warnoptions': [],
   'write_bytecode': 1,
   'xoptions': []}
msg353130 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-25 00:58
And on FreeBSD.

AMD64 FreeBSD CURRENT Shared 3.x:
https://buildbot.python.org/all/#/builders/168/builds/1510

======================================================================
FAIL: test_init_pybuilddir (test.test_embed.InitConfigTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 1130, in test_init_pybuilddir
    self.check_all_configs("test_init_compat_config", config,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[671 chars]ne, 'module_search_paths': ['/usr/home/buildbo[1297 chars]': 1} != {'_co[671 chars]ne, 'executable': '/tmp/tmpsa4mgwz2/_testembed[1285 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
-  'base_executable': '/var/tmp/tmpsa4mgwz2/_testembed',
?                      ----

+  'base_executable': '/tmp/tmpsa4mgwz2/_testembed',
   'base_prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
   'buffered_stdio': 1,
   'bytes_warning': 0,
   'check_hash_pycs_mode': 'default',
   'configure_c_stdio': 0,
   'dev_mode': 0,
   'dump_refs': 0,
   'exec_prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
-  'executable': '/var/tmp/tmpsa4mgwz2/_testembed',
?                 ----

+  'executable': '/tmp/tmpsa4mgwz2/_testembed',
   'faulthandler': 0,
   'hash_seed': 0,
   'home': None,
   'import_time': 0,
   'inspect': 0,
   'install_signal_handlers': 1,
   'interactive': 0,
   'isolated': 0,
   'malloc_stats': 0,
   'module_search_paths': ['/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python39.zip',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/build/lib.freebsd-13.0-CURRENT-amd64-3.9-pydebug',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python39.zip',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python3.9',
-                          '/var/tmp/tmpsa4mgwz2/libdir'],
?                           ----

+                          '/tmp/tmpsa4mgwz2/libdir'],
   'optimization_level': 0,
   'parse_argv': 0,
   'parser_debug': 0,
   'pathconfig_warnings': 1,
   'prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
   'program_name': './_testembed',
   'pycache_prefix': None,
   'pythonpath_env': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python39.zip:/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib:/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/build/lib.freebsd-13.0-CURRENT-amd64-3.9-pydebug',
   'quiet': 0,
   'run_command': None,
   'run_filename': None,
   'run_module': None,
   'show_alloc_count': 0,
   'show_ref_count': 0,
   'site_import': 1,
   'skip_source_first_line': 0,
   'tracemalloc': 0,
   'use_environment': 1,
   'use_hash_seed': 0,
   'user_site_directory': 1,
   'verbose': 0,
   'warnoptions': [],
   'write_bytecode': 1,
   'xoptions': []}

======================================================================
FAIL: test_init_pyvenv_cfg (test.test_embed.InitConfigTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 1182, in test_init_pyvenv_cfg
    self.check_all_configs("test_init_compat_config", config,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[671 chars]ne, 'module_search_paths': ['/usr/home/buildbo[1216 chars]': 1} != {'_co[671 chars]ne, 'executable': '/tmp/tmpszvtbyq4/_testembed[1208 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/tmp/tmpr0gyjdqs',
-  'base_executable': '/var/tmp/tmpszvtbyq4/_testembed',
?                      ----

+  'base_executable': '/tmp/tmpszvtbyq4/_testembed',
   'base_prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
   'buffered_stdio': 1,
   'bytes_warning': 0,
   'check_hash_pycs_mode': 'default',
   'configure_c_stdio': 0,
   'dev_mode': 0,
   'dump_refs': 0,
   'exec_prefix': '/tmp/tmpr0gyjdqs',
-  'executable': '/var/tmp/tmpszvtbyq4/_testembed',
?                 ----

+  'executable': '/tmp/tmpszvtbyq4/_testembed',
   'faulthandler': 0,
   'hash_seed': 0,
   'home': None,
   'import_time': 0,
   'inspect': 0,
   'install_signal_handlers': 1,
   'interactive': 0,
   'isolated': 0,
   'malloc_stats': 0,
   'module_search_paths': ['/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python39.zip',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/build/lib.freebsd-13.0-CURRENT-amd64-3.9-pydebug',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python39.zip',
                           '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python3.9',
                           '/tmp/tmpr0gyjdqs/lib/python3.9/lib-dynload'],
   'optimization_level': 0,
   'parse_argv': 0,
   'parser_debug': 0,
   'pathconfig_warnings': 1,
   'prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
   'program_name': './_testembed',
   'pycache_prefix': None,
   'pythonpath_env': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target/lib/python39.zip:/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib:/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/build/lib.freebsd-13.0-CURRENT-amd64-3.9-pydebug',
   'quiet': 0,
   'run_command': None,
   'run_filename': None,
   'run_module': None,
   'show_alloc_count': 0,
   'show_ref_count': 0,
   'site_import': 1,
   'skip_source_first_line': 0,
   'tracemalloc': 0,
   'use_environment': 1,
   'use_hash_seed': 0,
   'user_site_directory': 1,
   'verbose': 0,
   'warnoptions': [],
   'write_bytecode': 1,
   'xoptions': []}
msg353205 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-25 14:30
New changeset 00508a7407d7d300b487532e2271534b20e378a7 by Victor Stinner in branch 'master':
bpo-38234: Fix test_embed pathconfig tests (GH-16390)
https://github.com/python/cpython/commit/00508a7407d7d300b487532e2271534b20e378a7
msg353241 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 00:22
New changeset 8bf39b606ef7b02c0279a80789f3c4824b0da5e9 by Victor Stinner in branch 'master':
bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)
https://github.com/python/cpython/commit/8bf39b606ef7b02c0279a80789f3c4824b0da5e9
msg353254 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 01:15
New changeset 88feaecd46a8f427e30ef7ad8cfcddfe392a2402 by Victor Stinner in branch 'master':
bpo-38234: Complete init config documentation (GH-16404)
https://github.com/python/cpython/commit/88feaecd46a8f427e30ef7ad8cfcddfe392a2402
msg353256 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 02:01
New changeset 49d99f01e6e51acec5ca57a02e857f0796bc418b by Victor Stinner in branch 'master':
bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406)
https://github.com/python/cpython/commit/49d99f01e6e51acec5ca57a02e857f0796bc418b
msg353277 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 07:48
Oh, there is a regression:

vstinner@apu$ ./python
Python 3.9.0a0 (heads/master:88feaecd46, Sep 26 2019, 03:26:37) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _tkinter
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]

PyInit__tkinter() calls Py_GetProgramName(). But Python initialization no longer copies PyConfig path configuration into _Py_path_info (the global configuration). So Python has to recompute the path configuration which logs 3 warnings.
msg353304 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 13:52
New changeset 12f2f177fc483723406d7917194e7f655a20631b by Victor Stinner in branch 'master':
bpo-38234: Py_Initialize() sets global path configuration (GH-16421)
https://github.com/python/cpython/commit/12f2f177fc483723406d7917194e7f655a20631b
msg353306 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 14:17
New changeset 96c8475362acb41decd1d7db9243f328973e5de7 by Victor Stinner in branch '3.8':
[3.8] bpo-38234: Backport init path config changes from master (GH-16423)
https://github.com/python/cpython/commit/96c8475362acb41decd1d7db9243f328973e5de7
msg353309 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-26 14:29
> Oh, there is a regression: (...) PyInit__tkinter() calls Py_GetProgramName(). But Python initialization no longer copies PyConfig path configuration into _Py_path_info (the global configuration). So Python has to recompute the path configuration which logs 3 warnings.

It's fixed by commit 12f2f177fc483723406d7917194e7f655a20631b which adds new tests to prevent any regression.
msg353397 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-27 20:05
Oh, the main issue has been fixed and I added a lot of tests. So I remove the "release blocker" priority.

I would like to keep the issue open since I plan to do one more change: use PyWideStringList in _PyPathConfig for module_search_paths to support paths which contains DELIM (":" on Unix).
History
Date User Action Args
2019-09-27 20:05:11vstinnersetpriority: release blocker ->

messages: + msg353397
2019-09-27 09:43:14koobssetnosy: + koobs
2019-09-26 14:29:38vstinnersetmessages: + msg353309
2019-09-26 14:17:37vstinnersetmessages: + msg353306
2019-09-26 14:05:21vstinnersetpull_requests: + pull_request16002
2019-09-26 13:52:03vstinnersetmessages: + msg353304
2019-09-26 13:37:07pablogsalsetpriority: release blocker
2019-09-26 13:19:44vstinnersetpull_requests: + pull_request16000
2019-09-26 07:48:40vstinnersetmessages: + msg353277
2019-09-26 02:01:53vstinnersetmessages: + msg353256
2019-09-26 01:40:47vstinnersetpull_requests: + pull_request15988
2019-09-26 01:15:11vstinnersetmessages: + msg353254
2019-09-26 00:26:39vstinnersetpull_requests: + pull_request15985
2019-09-26 00:22:48vstinnersetmessages: + msg353241
2019-09-25 22:00:19vstinnersetpull_requests: + pull_request15984
2019-09-25 14:30:39vstinnersetmessages: + msg353205
2019-09-25 14:12:27vstinnersetpull_requests: + pull_request15972
2019-09-25 00:58:32vstinnersetmessages: + msg353130
2019-09-25 00:57:20vstinnersetmessages: + msg353129
2019-09-25 00:54:28vstinnersetmessages: + msg353128
2019-09-25 00:32:42vstinnersetpull_requests: + pull_request15947
2019-09-25 00:10:38vstinnersetmessages: + msg353124
2019-09-24 23:50:05vstinnersetpull_requests: + pull_request15946
2019-09-24 16:21:08vstinnersetmessages: + msg353099
2019-09-24 16:01:00vstinnersetpull_requests: + pull_request15937
2019-09-24 15:44:30vstinnersetmessages: + msg353096
2019-09-24 15:11:51vstinnersetpull_requests: + pull_request15936
2019-09-23 23:16:56miss-islingtonsetnosy: + miss-islington
messages: + msg353043
2019-09-23 22:56:37miss-islingtonsetpull_requests: + pull_request15917
2019-09-23 22:55:52vstinnersetmessages: + msg353042
2019-09-23 22:27:17vstinnersetpull_requests: + pull_request15916
2019-09-23 22:23:23vstinnersetpull_requests: + pull_request15915
2019-09-23 17:50:32vstinnersetmessages: + msg353037
2019-09-23 17:04:13vstinnersetstage: resolved -> patch review
pull_requests: + pull_request15913
2019-09-23 16:47:40vstinnersetmessages: + msg353035
2019-09-23 16:12:35vstinnersetstatus: closed -> open
resolution: fixed ->
messages: + msg353034
2019-09-23 15:55:22vstinnersetpull_requests: + pull_request15912
2019-09-23 15:39:54steve.dowersetmessages: + msg353032
2019-09-23 15:16:44vstinnersetmessages: + msg353031
2019-09-23 15:09:30steve.dowersetnosy: + steve.dower
messages: + msg353030
2019-09-21 00:13:48vstinnersetstatus: open -> closed
priority: release blocker -> (no value)
messages: + msg352915

resolution: fixed
stage: patch review -> resolved
2019-09-21 00:13:17vstinnersetmessages: + msg352914
2019-09-20 23:53:51vstinnersetpull_requests: + pull_request15895
2019-09-20 23:50:30miss-islingtonsetpull_requests: + pull_request15894
2019-09-20 23:50:19vstinnersetmessages: + msg352913
2019-09-20 23:02:59vstinnersetmessages: + msg352909
2019-09-20 22:39:40vstinnersetpull_requests: + pull_request15893
2019-09-20 16:57:05vstinnersetmessages: + msg352871
2019-09-20 15:43:33vstinnersetmessages: + msg352862
2019-09-20 15:41:04vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request15884
2019-09-20 15:24:50pablogsalsetmessages: + msg352859
2019-09-20 15:20:59vstinnersetpriority: normal -> release blocker
nosy: + lukasz.langa
messages: + msg352857

2019-09-20 15:19:54pablogsalcreate