➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_cmd_line test_utf8_mode test_warnings fail in all FreeBSD 3.x (3.8) buildbots
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: koobs, lukasz.langa, matrixise, ncoghlan, pablogsal, vstinner, xtreak
Priority: normal Keywords: 3.7regression, patch

Created on 2018-07-24 14:08 by pablogsal, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8867 closed vstinner, 2018-08-23 09:51
PR 8868 merged vstinner, 2018-08-23 10:06
PR 8870 merged vstinner, 2018-08-23 10:27
Messages (12)
msg322303 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-24 14:08 
test_cmd_line test_utf8_mode test_warnings fail in AMD64 FreeBSD CURRENT buildbots:

https://buildbot.python.org/all/#/builders/79/builds/186
https://buildbot.python.org/all/#/builders/60/builds/182

Sample error:

test_xoptions (test.test_cmd_line.CmdLineTest) ... ok
test_ignore_PYTHONHASHSEED (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test_ignore_PYTHONPATH (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test test_cmd_line failed
test_sys_flags_not_set (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
======================================================================
FAIL: test_non_ascii (test.test_cmd_line.CmdLineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_cmd_line.py", line 149, in test_non_ascii
    assert_python_ok('-c', command)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/support/script_helper.py", line 157, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/support/script_helper.py", line 143, in _assert_python
    res.fail(cmd_line)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/support/script_helper.py", line 84, in fail
    err))
AssertionError: Process return code is 1
command line: ['/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/python', '-X', 'faulthandler', '-I', '-c', "assert(ord('��') == 230)"]
stdout:
---
---
stderr:
---
Unable to decode the command from the command line:
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 12-13: surrogates not allowed
---
----------------------------------------------------------------------
Ran 43 tests in 17.680s
FAILED (failures=1, skipped=2)
3 tests failed again:
    test_cmd_line test_utf8_mode test_warnings


and

est_showwarning_not_callable (test.test_warnings._WarningsTests) ... ok
test_showwarnmsg_missing (test.test_warnings._WarningsTests) ... ok
test test_warnings failed
test_stderr_none (test.test_warnings._WarningsTests) ... ok
======================================================================
FAIL: test_nonascii (test.test_warnings.CEnvironmentVariableTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_warnings/__init__.py", line 1151, in test_nonascii
    "['ignore:Deprecaci��nWarning']".encode('utf-8'))
AssertionError: b"['ignore:Deprecaci\\udcc3\\udcb3nWarning']" != b"['ignore:Deprecaci\xc3\xb3nWarning']"
======================================================================
FAIL: test_nonascii (test.test_warnings.PyEnvironmentVariableTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_warnings/__init__.py", line 1151, in test_nonascii
    "['ignore:Deprecaci��nWarning']".encode('utf-8'))
AssertionError: b"['ignore:Deprecaci\\udcc3\\udcb3nWarning']" != b"['ignore:Deprecaci\xc3\xb3nWarning']"
----------------------------------------------------------------------
Ran 111 tests in 4.635s
FAILED (failures=2)
3 tests failed again:
    test_cmd_line test_utf8_mode test_warnings
== Tests result: FAILURE then FAILURE ==
msg323036 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-08-03 07:12 
I tried the tests on FreeBSD 11.x and FreeBSD 10.x digitalocean droplets with master branch and the tests pass. Adding information that might be helpful.

# FreeBSD 11.2-RELEASE

root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # uname -a
FreeBSD freebsd-s-1vcpu-1gb-blr1-01 11.2-RELEASE FreeBSD 11.2-RELEASE #0 r335510: Fri Jun 22 04:32:14 UTC 2018     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python
Python 3.8.0a0 (heads/master:b6efc2c, Aug  3 2018, 06:26:53)
[Clang 6.0.0 (tags/RELEASE_600/final 326565)] on freebsd11
Type "help", "copyright", "credits" or "license" for more information.
>>>
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # setenv PYTHONIOENCODING utf-8
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # setenv PYTHONWARNINGS ignore:DeprecaciÛnWarning
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # setenv PYTHONDEVMODE ""
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python -c "import sys; sys.stdout.write(str(sys.warnoptions))"
['ignore:DeprecaciÛnWarning']
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python -m unittest -v test.test_warnings test.test_cmd_line test.test_utf8_mode
test_issue_8766 (test.test_warnings.BootstrapTest) ... ok
test_catch_warnings_defaults (test.test_warnings.CCatchWarningTests) ... ok
test_catch_warnings_recording (test.test_warnings.CCatchWarningTests) ... ok
test_catch_warnings_reentry_guard (test.test_warnings.CCatchWarningTests) ... ok
test_catch_warnings_restore (test.test_warnings.CCatchWarningTests) ... ok
test_check_warnings (test.test_warnings.CCatchWarningTests) ... ok
test_record_override_showwarning_before (test.test_warnings.CCatchWarningTests) ... ok
test_record_override_showwarning_inside (test.test_warnings.CCatchWarningTests) ... ok
test_comma_separated_warnings (test.test_warnings.CEnvironmentVariableTests) ... ok
test_conflicting_envvar_and_command_line (test.test_warnings.CEnvironmentVariableTests) ... ok
test_default_filter_configuration (test.test_warnings.CEnvironmentVariableTests) ... ok
test_envvar_and_command_line (test.test_warnings.CEnvironmentVariableTests) ... ok
test_nonascii (test.test_warnings.CEnvironmentVariableTests) ... ok
test_single_warning (test.test_warnings.CEnvironmentVariableTests) ... ok
test_always (test.test_warnings.CFilterTests) ... ok
test_always_after_default (test.test_warnings.CFilterTests) ... ok
test_append_duplicate (test.test_warnings.CFilterTests) ... ok
test_default (test.test_warnings.CFilterTests) ... ok
test_error (test.test_warnings.CFilterTests) ... ok
test_error_after_default (test.test_warnings.CFilterTests) ... ok
test_filterwarnings (test.test_warnings.CFilterTests) ... ok
test_filterwarnings_duplicate_filters (test.test_warnings.CFilterTests) ... ok
test_ignore (test.test_warnings.CFilterTests) ... ok
test_ignore_after_default (test.test_warnings.CFilterTests) ... ok
test_inheritance (test.test_warnings.CFilterTests) ... ok
test_message_matching (test.test_warnings.CFilterTests) ... ok
test_module (test.test_warnings.CFilterTests) ... ok
test_module_globals (test.test_warnings.CFilterTests) ... ok
test_mutate_filter_list (test.test_warnings.CFilterTests) ... ok
test_once (test.test_warnings.CFilterTests) ... ok
test_ordering (test.test_warnings.CFilterTests) ... ok
test_simplefilter_duplicate_filters (test.test_warnings.CFilterTests) ... ok
test_module_all_attribute (test.test_warnings.CPublicAPITests) ... ok
test_improper_input (test.test_warnings.CWCmdLineTests) ... ok
test_accelerated (test.test_warnings.CWarnTests) ... ok
test_bad_str (test.test_warnings.CWarnTests) ... ok
test_exec_filename (test.test_warnings.CWarnTests) ... ok
test_filename (test.test_warnings.CWarnTests) ... ok
test_message (test.test_warnings.CWarnTests) ... ok
test_stacklevel (test.test_warnings.CWarnTests) ... ok
test_stacklevel_import (test.test_warnings.CWarnTests) ... ok
test_warn_explicit_non_ascii_filename (test.test_warnings.CWarnTests) ... ok
test_warn_explicit_type_errors (test.test_warnings.CWarnTests) ... ok
test_warn_nonstandard_types (test.test_warnings.CWarnTests) ... ok
test_warning_classes (test.test_warnings.CWarnTests) ... ok
test_formatwarning (test.test_warnings.CWarningsDisplayTests) ... ok
test_showwarning (test.test_warnings.CWarningsDisplayTests) ... ok
test_finalization (test.test_warnings.FinalizationTest) ... ok
test_late_resource_warning (test.test_warnings.FinalizationTest) ... ok
test_catch_warnings_defaults (test.test_warnings.PyCatchWarningTests) ... ok
test_catch_warnings_recording (test.test_warnings.PyCatchWarningTests) ... ok
test_catch_warnings_reentry_guard (test.test_warnings.PyCatchWarningTests) ... ok
test_catch_warnings_restore (test.test_warnings.PyCatchWarningTests) ... ok
test_check_warnings (test.test_warnings.PyCatchWarningTests) ... ok
test_record_override_showwarning_before (test.test_warnings.PyCatchWarningTests) ... ok
test_record_override_showwarning_inside (test.test_warnings.PyCatchWarningTests) ... ok
test_comma_separated_warnings (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_conflicting_envvar_and_command_line (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_default_filter_configuration (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_envvar_and_command_line (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_nonascii (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_single_warning (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_always (test.test_warnings.PyFilterTests) ... ok
test_always_after_default (test.test_warnings.PyFilterTests) ... ok
test_append_duplicate (test.test_warnings.PyFilterTests) ... ok
test_default (test.test_warnings.PyFilterTests) ... ok
test_error (test.test_warnings.PyFilterTests) ... ok
test_error_after_default (test.test_warnings.PyFilterTests) ... ok
test_filterwarnings (test.test_warnings.PyFilterTests) ... ok
test_filterwarnings_duplicate_filters (test.test_warnings.PyFilterTests) ... ok
test_ignore (test.test_warnings.PyFilterTests) ... ok
test_ignore_after_default (test.test_warnings.PyFilterTests) ... ok
test_inheritance (test.test_warnings.PyFilterTests) ... ok
test_message_matching (test.test_warnings.PyFilterTests) ... ok
test_module (test.test_warnings.PyFilterTests) ... ok
test_module_globals (test.test_warnings.PyFilterTests) ... ok
test_mutate_filter_list (test.test_warnings.PyFilterTests) ... ok
test_once (test.test_warnings.PyFilterTests) ... ok
test_ordering (test.test_warnings.PyFilterTests) ... ok
test_simplefilter_duplicate_filters (test.test_warnings.PyFilterTests) ... ok
test_module_all_attribute (test.test_warnings.PyPublicAPITests) ... ok
test_improper_input (test.test_warnings.PyWCmdLineTests) ... ok
test_improper_option (test.test_warnings.PyWCmdLineTests) ... ok
test_warnings_bootstrap (test.test_warnings.PyWCmdLineTests) ... ok
test_bad_str (test.test_warnings.PyWarnTests) ... ok
test_exec_filename (test.test_warnings.PyWarnTests) ... ok
test_filename (test.test_warnings.PyWarnTests) ... ok
test_message (test.test_warnings.PyWarnTests) ... ok
test_pure_python (test.test_warnings.PyWarnTests) ... ok
test_stacklevel (test.test_warnings.PyWarnTests) ... ok
test_stacklevel_import (test.test_warnings.PyWarnTests) ... ok
test_warn_explicit_non_ascii_filename (test.test_warnings.PyWarnTests) ... ok
test_warn_explicit_type_errors (test.test_warnings.PyWarnTests) ... ok
test_warn_nonstandard_types (test.test_warnings.PyWarnTests) ... ok
test_warning_classes (test.test_warnings.PyWarnTests) ... ok
test_formatwarning (test.test_warnings.PyWarningsDisplayTests) ... ok
test_showwarning (test.test_warnings.PyWarningsDisplayTests) ... ok
test_tracemalloc (test.test_warnings.PyWarningsDisplayTests) ... ok
test_default_action (test.test_warnings._WarningsTests) ... ok
test_filename_none (test.test_warnings._WarningsTests) ... ok
test_filter (test.test_warnings._WarningsTests) ... ok
test_issue31285 (test.test_warnings._WarningsTests) ... ok
test_issue31411 (test.test_warnings._WarningsTests) ... ok
test_issue31416 (test.test_warnings._WarningsTests) ... ok
test_issue31566 (test.test_warnings._WarningsTests) ... ok
test_onceregistry (test.test_warnings._WarningsTests) ... ok
test_show_warning_output (test.test_warnings._WarningsTests) ... ok
test_showwarning_missing (test.test_warnings._WarningsTests) ... ok
test_showwarning_not_callable (test.test_warnings._WarningsTests) ... ok
test_showwarnmsg_missing (test.test_warnings._WarningsTests) ... ok
test_stderr_none (test.test_warnings._WarningsTests) ... ok
test_argv0_normalization (test.test_cmd_line.CmdLineTest) ... skipped 'bpo-32457 only applies on Windows'
test_builtin_input (test.test_cmd_line.CmdLineTest) ... ok
test_closed_stdout (test.test_cmd_line.CmdLineTest) ... ok
test_del___main__ (test.test_cmd_line.CmdLineTest) ... ok
test_directories (test.test_cmd_line.CmdLineTest) ... ok
test_displayhook_unencodable (test.test_cmd_line.CmdLineTest) ... ok
test_empty_PYTHONPATH_issue16309 (test.test_cmd_line.CmdLineTest) ... ok
test_hash_randomization (test.test_cmd_line.CmdLineTest) ... ok
test_isolatedmode (test.test_cmd_line.CmdLineTest) ... ok
test_large_PYTHONPATH (test.test_cmd_line.CmdLineTest) ... ok
test_no_std_streams (test.test_cmd_line.CmdLineTest) ... ok
test_no_stderr (test.test_cmd_line.CmdLineTest) ... ok
test_no_stdin (test.test_cmd_line.CmdLineTest) ... ok
test_no_stdout (test.test_cmd_line.CmdLineTest) ... ok
test_non_ascii (test.test_cmd_line.CmdLineTest) ... ok
test_optimize (test.test_cmd_line.CmdLineTest) ... ok
test_osx_android_utf8 (test.test_cmd_line.CmdLineTest) ... skipped 'test specific to Mac OS X and Android'
test_output_newline (test.test_cmd_line.CmdLineTest) ... ok
test_pythondevmode_env (test.test_cmd_line.CmdLineTest) ... ok
test_pythonmalloc (test.test_cmd_line.CmdLineTest) ... ok
test_run_code (test.test_cmd_line.CmdLineTest) ... ok
test_run_module (test.test_cmd_line.CmdLineTest) ... ok
test_run_module_bug1764407 (test.test_cmd_line.CmdLineTest) ... ok
test_set_pycache_prefix (test.test_cmd_line.CmdLineTest) ... ok
test_showrefcount (test.test_cmd_line.CmdLineTest) ... ok
test_site_flag (test.test_cmd_line.CmdLineTest) ... ok
test_stdin_readline (test.test_cmd_line.CmdLineTest) ... ok
test_stdout_flush_at_shutdown (test.test_cmd_line.CmdLineTest) ... ok
test_sys_flags_set (test.test_cmd_line.CmdLineTest) ... ok
test_unbuffered_input (test.test_cmd_line.CmdLineTest) ... ok
test_unbuffered_output (test.test_cmd_line.CmdLineTest) ... ok
test_undecodable_code (test.test_cmd_line.CmdLineTest) ... ok
test_unknown_options (test.test_cmd_line.CmdLineTest) ... ok
test_unmached_quote (test.test_cmd_line.CmdLineTest) ... ok
test_usage (test.test_cmd_line.CmdLineTest) ... ok
test_verbose (test.test_cmd_line.CmdLineTest) ... ok
test_version (test.test_cmd_line.CmdLineTest) ... ok
test_warnings_filter_precedence (test.test_cmd_line.CmdLineTest) ... ok
test_xdev (test.test_cmd_line.CmdLineTest) ... ok
test_xoptions (test.test_cmd_line.CmdLineTest) ... ok
test_ignore_PYTHONHASHSEED (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test_ignore_PYTHONPATH (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test_sys_flags_not_set (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test_cmd_line (test.test_utf8_mode.UTF8ModeTests) ... ok
test_env_var (test.test_utf8_mode.UTF8ModeTests) ... ok
test_filesystemencoding (test.test_utf8_mode.UTF8ModeTests) ... ok
test_io (test.test_utf8_mode.UTF8ModeTests) ... ok
test_io_encoding (test.test_utf8_mode.UTF8ModeTests) ... ok
test_locale_getpreferredencoding (test.test_utf8_mode.UTF8ModeTests) ... ok
test_optim_level (test.test_utf8_mode.UTF8ModeTests) ... ok
test_posix_locale (test.test_utf8_mode.UTF8ModeTests) ... ok
test_stdio (test.test_utf8_mode.UTF8ModeTests) ... ok
test_xoption (test.test_utf8_mode.UTF8ModeTests) ... ok

----------------------------------------------------------------------
Ran 164 tests in 6.439s

# FreeBSD 10.4-RELEASE

root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # uname -a
FreeBSD freebsd-s-1vcpu-1gb-blr1-01 10.4-RELEASE FreeBSD 10.4-RELEASE #0 r324094: Fri Sep 29 01:45:44 UTC 2017     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd6
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python
Python 3.8.0a0 (heads/master:b6efc2c, Aug  3 2018, 06:58:22)
[Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>>
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # setenv PYTHONIOENCODING utf-8
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # setenv PYTHONWARNINGS ignore:DeprecaciÛnWarning
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # setenv PYTHONDEVMODE ""
root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python -c "import sys; sys.stdout.write(str(sys.warnoptions))"
['ignore:DeprecaciÛnWarning']

# Ran as a separate session on the same machine to ignore environment variables

root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python -m unittest -v test.test_warnings test.test_cmd_line test.test_utf8_mode
test_issue_8766 (test.test_warnings.BootstrapTest) ... ok
test_catch_warnings_defaults (test.test_warnings.CCatchWarningTests) ... ok
test_catch_warnings_recording (test.test_warnings.CCatchWarningTests) ... ok
test_catch_warnings_reentry_guard (test.test_warnings.CCatchWarningTests) ... ok
test_catch_warnings_restore (test.test_warnings.CCatchWarningTests) ... ok
test_check_warnings (test.test_warnings.CCatchWarningTests) ... ok
test_record_override_showwarning_before (test.test_warnings.CCatchWarningTests) ... ok
test_record_override_showwarning_inside (test.test_warnings.CCatchWarningTests) ... ok
test_comma_separated_warnings (test.test_warnings.CEnvironmentVariableTests) ... ok
test_conflicting_envvar_and_command_line (test.test_warnings.CEnvironmentVariableTests) ... ok
test_default_filter_configuration (test.test_warnings.CEnvironmentVariableTests) ... ok
test_envvar_and_command_line (test.test_warnings.CEnvironmentVariableTests) ... ok
test_nonascii (test.test_warnings.CEnvironmentVariableTests) ... ok
test_single_warning (test.test_warnings.CEnvironmentVariableTests) ... ok
test_always (test.test_warnings.CFilterTests) ... ok
test_always_after_default (test.test_warnings.CFilterTests) ... ok
test_append_duplicate (test.test_warnings.CFilterTests) ... ok
test_default (test.test_warnings.CFilterTests) ... ok
test_error (test.test_warnings.CFilterTests) ... ok
test_error_after_default (test.test_warnings.CFilterTests) ... ok
test_filterwarnings (test.test_warnings.CFilterTests) ... ok
test_filterwarnings_duplicate_filters (test.test_warnings.CFilterTests) ... ok
test_ignore (test.test_warnings.CFilterTests) ... ok
test_ignore_after_default (test.test_warnings.CFilterTests) ... ok
test_inheritance (test.test_warnings.CFilterTests) ... ok
test_message_matching (test.test_warnings.CFilterTests) ... ok
test_module (test.test_warnings.CFilterTests) ... ok
test_module_globals (test.test_warnings.CFilterTests) ... ok
test_mutate_filter_list (test.test_warnings.CFilterTests) ... ok
test_once (test.test_warnings.CFilterTests) ... ok
test_ordering (test.test_warnings.CFilterTests) ... ok
test_simplefilter_duplicate_filters (test.test_warnings.CFilterTests) ... ok
test_module_all_attribute (test.test_warnings.CPublicAPITests) ... ok
test_improper_input (test.test_warnings.CWCmdLineTests) ... ok
test_accelerated (test.test_warnings.CWarnTests) ... ok
test_bad_str (test.test_warnings.CWarnTests) ... ok
test_exec_filename (test.test_warnings.CWarnTests) ... ok
test_filename (test.test_warnings.CWarnTests) ... ok
test_message (test.test_warnings.CWarnTests) ... ok
test_stacklevel (test.test_warnings.CWarnTests) ... ok
test_stacklevel_import (test.test_warnings.CWarnTests) ... ok
test_warn_explicit_non_ascii_filename (test.test_warnings.CWarnTests) ... ok
test_warn_explicit_type_errors (test.test_warnings.CWarnTests) ... ok
test_warn_nonstandard_types (test.test_warnings.CWarnTests) ... ok
test_warning_classes (test.test_warnings.CWarnTests) ... ok
test_formatwarning (test.test_warnings.CWarningsDisplayTests) ... ok
test_showwarning (test.test_warnings.CWarningsDisplayTests) ... ok
test_finalization (test.test_warnings.FinalizationTest) ... ok
test_late_resource_warning (test.test_warnings.FinalizationTest) ... ok
test_catch_warnings_defaults (test.test_warnings.PyCatchWarningTests) ... ok
test_catch_warnings_recording (test.test_warnings.PyCatchWarningTests) ... ok
test_catch_warnings_reentry_guard (test.test_warnings.PyCatchWarningTests) ... ok
test_catch_warnings_restore (test.test_warnings.PyCatchWarningTests) ... ok
test_check_warnings (test.test_warnings.PyCatchWarningTests) ... ok
test_record_override_showwarning_before (test.test_warnings.PyCatchWarningTests) ... ok
test_record_override_showwarning_inside (test.test_warnings.PyCatchWarningTests) ... ok
test_comma_separated_warnings (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_conflicting_envvar_and_command_line (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_default_filter_configuration (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_envvar_and_command_line (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_nonascii (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_single_warning (test.test_warnings.PyEnvironmentVariableTests) ... ok
test_always (test.test_warnings.PyFilterTests) ... ok
test_always_after_default (test.test_warnings.PyFilterTests) ... ok
test_append_duplicate (test.test_warnings.PyFilterTests) ... ok
test_default (test.test_warnings.PyFilterTests) ... ok
test_error (test.test_warnings.PyFilterTests) ... ok
test_error_after_default (test.test_warnings.PyFilterTests) ... ok
test_filterwarnings (test.test_warnings.PyFilterTests) ... ok
test_filterwarnings_duplicate_filters (test.test_warnings.PyFilterTests) ... ok
test_ignore (test.test_warnings.PyFilterTests) ... ok
test_ignore_after_default (test.test_warnings.PyFilterTests) ... ok
test_inheritance (test.test_warnings.PyFilterTests) ... ok
test_message_matching (test.test_warnings.PyFilterTests) ... ok
test_module (test.test_warnings.PyFilterTests) ... ok
test_module_globals (test.test_warnings.PyFilterTests) ... ok
test_mutate_filter_list (test.test_warnings.PyFilterTests) ... ok
test_once (test.test_warnings.PyFilterTests) ... ok
test_ordering (test.test_warnings.PyFilterTests) ... ok
test_simplefilter_duplicate_filters (test.test_warnings.PyFilterTests) ... ok
test_module_all_attribute (test.test_warnings.PyPublicAPITests) ... ok
test_improper_input (test.test_warnings.PyWCmdLineTests) ... ok
test_improper_option (test.test_warnings.PyWCmdLineTests) ... ok
test_warnings_bootstrap (test.test_warnings.PyWCmdLineTests) ... ok
test_bad_str (test.test_warnings.PyWarnTests) ... ok
test_exec_filename (test.test_warnings.PyWarnTests) ... ok
test_filename (test.test_warnings.PyWarnTests) ... ok
test_message (test.test_warnings.PyWarnTests) ... ok
test_pure_python (test.test_warnings.PyWarnTests) ... ok
test_stacklevel (test.test_warnings.PyWarnTests) ... ok
test_stacklevel_import (test.test_warnings.PyWarnTests) ... ok
test_warn_explicit_non_ascii_filename (test.test_warnings.PyWarnTests) ... ok
test_warn_explicit_type_errors (test.test_warnings.PyWarnTests) ... ok
test_warn_nonstandard_types (test.test_warnings.PyWarnTests) ... ok
test_warning_classes (test.test_warnings.PyWarnTests) ... ok
test_formatwarning (test.test_warnings.PyWarningsDisplayTests) ... ok
test_showwarning (test.test_warnings.PyWarningsDisplayTests) ... ok
test_tracemalloc (test.test_warnings.PyWarningsDisplayTests) ... ok
test_default_action (test.test_warnings._WarningsTests) ... ok
test_filename_none (test.test_warnings._WarningsTests) ... ok
test_filter (test.test_warnings._WarningsTests) ... ok
test_issue31285 (test.test_warnings._WarningsTests) ... ok
test_issue31411 (test.test_warnings._WarningsTests) ... ok
test_issue31416 (test.test_warnings._WarningsTests) ... ok
test_issue31566 (test.test_warnings._WarningsTests) ... ok
test_onceregistry (test.test_warnings._WarningsTests) ... ok
test_show_warning_output (test.test_warnings._WarningsTests) ... ok
test_showwarning_missing (test.test_warnings._WarningsTests) ... ok
test_showwarning_not_callable (test.test_warnings._WarningsTests) ... ok
test_showwarnmsg_missing (test.test_warnings._WarningsTests) ... ok
test_stderr_none (test.test_warnings._WarningsTests) ... ok
test_argv0_normalization (test.test_cmd_line.CmdLineTest) ... skipped 'bpo-32457 only applies on Windows'
test_builtin_input (test.test_cmd_line.CmdLineTest) ... ok
test_closed_stdout (test.test_cmd_line.CmdLineTest) ... ok
test_del___main__ (test.test_cmd_line.CmdLineTest) ... ok
test_directories (test.test_cmd_line.CmdLineTest) ... ok
test_displayhook_unencodable (test.test_cmd_line.CmdLineTest) ... ok
test_empty_PYTHONPATH_issue16309 (test.test_cmd_line.CmdLineTest) ... ok
test_hash_randomization (test.test_cmd_line.CmdLineTest) ... ok
test_isolatedmode (test.test_cmd_line.CmdLineTest) ... ok
test_large_PYTHONPATH (test.test_cmd_line.CmdLineTest) ... ok
test_no_std_streams (test.test_cmd_line.CmdLineTest) ... ok
test_no_stderr (test.test_cmd_line.CmdLineTest) ... ok
test_no_stdin (test.test_cmd_line.CmdLineTest) ... ok
test_no_stdout (test.test_cmd_line.CmdLineTest) ... ok
test_non_ascii (test.test_cmd_line.CmdLineTest) ... ok
test_optimize (test.test_cmd_line.CmdLineTest) ... ok
test_osx_android_utf8 (test.test_cmd_line.CmdLineTest) ... skipped 'test specific to Mac OS X and Android'
test_output_newline (test.test_cmd_line.CmdLineTest) ... ok
test_pythondevmode_env (test.test_cmd_line.CmdLineTest) ... ok
test_pythonmalloc (test.test_cmd_line.CmdLineTest) ... ok
test_run_code (test.test_cmd_line.CmdLineTest) ... ok
test_run_module (test.test_cmd_line.CmdLineTest) ... ok
test_run_module_bug1764407 (test.test_cmd_line.CmdLineTest) ... ok
test_set_pycache_prefix (test.test_cmd_line.CmdLineTest) ... ok
test_showrefcount (test.test_cmd_line.CmdLineTest) ... ok
test_site_flag (test.test_cmd_line.CmdLineTest) ... ok
test_stdin_readline (test.test_cmd_line.CmdLineTest) ... ok
test_stdout_flush_at_shutdown (test.test_cmd_line.CmdLineTest) ... ok
test_sys_flags_set (test.test_cmd_line.CmdLineTest) ... ok
test_unbuffered_input (test.test_cmd_line.CmdLineTest) ... ok
test_unbuffered_output (test.test_cmd_line.CmdLineTest) ... ok
test_undecodable_code (test.test_cmd_line.CmdLineTest) ... ok
test_unknown_options (test.test_cmd_line.CmdLineTest) ... ok
test_unmached_quote (test.test_cmd_line.CmdLineTest) ... ok
test_usage (test.test_cmd_line.CmdLineTest) ... ok
test_verbose (test.test_cmd_line.CmdLineTest) ... ok
test_version (test.test_cmd_line.CmdLineTest) ... ok
test_warnings_filter_precedence (test.test_cmd_line.CmdLineTest) ... ok
test_xdev (test.test_cmd_line.CmdLineTest) ... ok
test_xoptions (test.test_cmd_line.CmdLineTest) ... ok
test_ignore_PYTHONHASHSEED (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test_ignore_PYTHONPATH (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test_sys_flags_not_set (test.test_cmd_line.IgnoreEnvironmentTest) ... ok
test_cmd_line (test.test_utf8_mode.UTF8ModeTests) ... ok
test_env_var (test.test_utf8_mode.UTF8ModeTests) ... ok
test_filesystemencoding (test.test_utf8_mode.UTF8ModeTests) ... ok
test_io (test.test_utf8_mode.UTF8ModeTests) ... ok
test_io_encoding (test.test_utf8_mode.UTF8ModeTests) ... ok
test_locale_getpreferredencoding (test.test_utf8_mode.UTF8ModeTests) ... ok
test_optim_level (test.test_utf8_mode.UTF8ModeTests) ... ok
test_posix_locale (test.test_utf8_mode.UTF8ModeTests) ... ok
test_stdio (test.test_utf8_mode.UTF8ModeTests) ... ok
test_xoption (test.test_utf8_mode.UTF8ModeTests) ... ok

----------------------------------------------------------------------
Ran 164 tests in 15.137s

OK (skipped=2)



Thanks
msg323037 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-08-03 07:18 
xtreak

in this case, it's an issue on the buildbot cluster, we have to be sure that we don't have this error again.

and for that, we have to find the origin of the crash on buildbot.
msg323832 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-08-21 15:00 
This is definitely dependent on the environment. I can reproduce exactly the same failures on a CentOS 7 box at work, too.
msg323833 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-08-21 15:18 
I think the root cause is this: the mentioned tests are invalid when no LC_* or LANG environment variables are present. Michael's sleuthing in issue34347 might provide a way for us to fix those.

As a workaround in the mean time, if this is an option on your system, set LC_ALL=en_US.UTF8, the three failing tests pass now.
msg323834 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-08-21 15:20 
Looking at what's going on here, this is quite possibly a PEP 538 side-effect (I see surrogate escapes).
msg323880 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-08-22 13:50 
Nothing has changed recently in the PEP 538 implementation itself, and both those builders were green last month:

* https://buildbot.python.org/all/#/builders/60/builds/181
* https://buildbot.python.org/all/#/builders/79/builds/185

The first failing debug build was https://buildbot.python.org/all/#/builders/60/builds/182 which rearranged some parts of the startup sequence: https://github.com/python/cpython/commit/d19d8d5279f156bc8f6736b5f16f069879b9519b

Similarly, the 3.7 builds started failing in https://buildbot.python.org/all/#/builders/124/builds/489, which made Py_Initialize work more closely to the way it worked in Python 3.6: https://github.com/python/cpython/commit/0c90d6f75931da4fec84d06c2efe9dd94bb96b77

So given the output reported in the current build failures, my guess would be that something has gotten out of sequence in the code that has to handle re-encoding and decoding command line values and environment variables that have initially been decoded incorrectly (i.e. as surrogate escaped ASCII).
msg323932 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 10:06 
I confirm that "LC_ALL= LANG= LC_CTYPE= ./python -m test -v test_cmd_line -m test_non_ascii" fails on FreeBSD on the master branch, whereas it pass  before commit d19d8d5279f156bc8f6736b5f16f069879b9519b. This commit introduced the regression.

Thanks for the bisection Nick!

My change avoids modifying global configurable variables, whereas Py_DecodeLocale() and Py_EncodeLocale() depend on Py_UTF8Mode.
msg323935 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 10:18 
I wrote PR 8867 but it doesn't fix all cases, so I abandonned it.

Then I wrote PR 8868 which sets Py_UTF8Mode in pymain_read_conf() and should fix the issue.

The root issue is that we have C functions which depend on the global configuration variables like Py_UTF8Mode :-(
msg323937 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 10:23 
New changeset 89487f51b8d6ba8a55f5de0ed689e46fefe73cc9 by Victor Stinner in branch 'master':
bpo-34207: Fix pymain_read_conf() for UTF-8 Mode (GH-8868)
https://github.com/python/cpython/commit/89487f51b8d6ba8a55f5de0ed689e46fefe73cc9
msg323939 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 10:41 
New changeset 80a0ebaa8385988b1d2dbbbb71ed1f548fc32ad7 by Victor Stinner in branch '3.7':
bpo-34207: Fix pymain_read_conf() for UTF-8 Mode (GH-8868) (GH-8870)
https://github.com/python/cpython/commit/80a0ebaa8385988b1d2dbbbb71ed1f548fc32ad7
msg323940 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 10:43 
The regression should now be fixed in 3.7 and master branches.
History
Date User Action Args
2022-04-11 14:59:03adminsetgithub: 78388
2018-08-28 21:10:39vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-08-23 10:43:50vstinnersetmessages: + msg323940
2018-08-23 10:41:39vstinnersetmessages: + msg323939
2018-08-23 10:27:34vstinnersetpull_requests: + pull_request8346
2018-08-23 10:23:48vstinnersetmessages: + msg323937
2018-08-23 10:18:52vstinnersetmessages: + msg323935
2018-08-23 10:06:50vstinnersetpull_requests: + pull_request8343
2018-08-23 10:06:35vstinnersetmessages: + msg323932
2018-08-23 09:51:12vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request8341
2018-08-22 13:50:31ncoghlansetnosy: + vstinner
messages: + msg323880
2018-08-21 15:20:18lukasz.langasetnosy: + ncoghlan
2018-08-21 15:20:05lukasz.langasetmessages: + msg323834
2018-08-21 15:18:06lukasz.langasetmessages: + msg323833
2018-08-21 15:00:28lukasz.langasetnosy: + lukasz.langa
messages: + msg323832
2018-08-21 14:56:45lukasz.langasetkeywords: + 3.7regression
versions: + Python 3.7
2018-08-21 14:56:22lukasz.langalinkissue34403 dependencies
2018-08-21 14:56:17lukasz.langalinkissue34347 dependencies
2018-08-03 07:18:43matrixisesetnosy: + matrixise
messages: + msg323037
2018-08-03 07:12:12xtreaksetmessages: + msg323036
2018-07-31 03:19:28koobssetnosy: + koobs

title: test_cmd_line test_utf8_mode test_warnings fail in AMD64 FreeBSD CURRENT buildbots -> test_cmd_line test_utf8_mode test_warnings fail in all FreeBSD 3.x (3.8) buildbots
2018-07-24 14:57:38xtreaksetnosy: + xtreak
2018-07-24 14:08:35pablogsalcreate