test_cmd_line test_utf8_mode test_warnings fail in all FreeBSD 3.x (3.8) buildbots #78388
Comments
|
test_cmd_line test_utf8_mode test_warnings fail in AMD64 FreeBSD CURRENT buildbots: https://buildbot.python.org/all/#/builders/79/builds/186 Sample error: test_xoptions (test.test_cmd_line.CmdLineTest) ... ok Traceback (most recent call last):
File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_cmd_line.py", line 149, in test_non_ascii
assert_python_ok('-c', command)
File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/support/script_helper.py", line 157, in assert_python_ok
return _assert_python(True, *args, **env_vars)
File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/support/script_helper.py", line 143, in _assert_python
res.fail(cmd_line)
File "/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/support/script_helper.py", line 84, in fail
err))
AssertionError: Process return code is 1
command line: ['/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/python', '-X', 'faulthandler', '-I', '-c', "assert(ord('��') == 230)"]
stdout:
---
stderr: and est_showwarning_not_callable (test.test_warnings._WarningsTests) ... ok Traceback (most recent call last):
File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_warnings/__init__.py", line 1151, in test_nonascii
"['ignore:Deprecaci��nWarning']".encode('utf-8'))
AssertionError: b"['ignore:Deprecaci\\udcc3\\udcb3nWarning']" != b"['ignore:Deprecaci\xc3\xb3nWarning']"====================================================================== Traceback (most recent call last):
File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_warnings/__init__.py", line 1151, in test_nonascii
"['ignore:Deprecaci��nWarning']".encode('utf-8'))
AssertionError: b"['ignore:Deprecaci\\udcc3\\udcb3nWarning']" != b"['ignore:Deprecaci\xc3\xb3nWarning']"Ran 111 tests in 4.635s |
pablogsal
added
3.8
koobs
changed the title
|
I tried the tests on FreeBSD 11.x and FreeBSD 10.x digitalocean droplets with master branch and the tests pass. Adding information that might be helpful. # FreeBSD 11.2-RELEASE root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # uname -a root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python
---------------------------------------------------------------------- # FreeBSD 10.4-RELEASE root@freebsd-s-1vcpu-1gb-blr1-01:
# Ran as a separate session on the same machine to ignore environment variables root@freebsd-s-1vcpu-1gb-blr1-01:~/cpython # ./python -m unittest -v test.test_warnings test.test_cmd_line test.test_utf8_mode ---------------------------------------------------------------------- OK (skipped=2) Thanks |
|
xtreak in this case, it's an issue on the buildbot cluster, we have to be sure that we don't have this error again. and for that, we have to find the origin of the crash on buildbot. |
|
This is definitely dependent on the environment. I can reproduce exactly the same failures on a CentOS 7 box at work, too. |
|
I think the root cause is this: the mentioned tests are invalid when no LC_* or LANG environment variables are present. Michael's sleuthing in bpo-34347 might provide a way for us to fix those. As a workaround in the mean time, if this is an option on your system, set LC_ALL=en_US.UTF8, the three failing tests pass now. |
|
Looking at what's going on here, this is quite possibly a PEP-538 side-effect (I see surrogate escapes). |
|
Nothing has changed recently in the PEP-538 implementation itself, and both those builders were green last month:
The first failing debug build was https://buildbot.python.org/all/#/builders/60/builds/182 which rearranged some parts of the startup sequence: d19d8d5 Similarly, the 3.7 builds started failing in https://buildbot.python.org/all/#/builders/124/builds/489, which made Py_Initialize work more closely to the way it worked in Python 3.6: 0c90d6f So given the output reported in the current build failures, my guess would be that something has gotten out of sequence in the code that has to handle re-encoding and decoding command line values and environment variables that have initially been decoded incorrectly (i.e. as surrogate escaped ASCII). |
|
I confirm that "LC_ALL= LANG= LC_CTYPE= ./python -m test -v test_cmd_line -m test_non_ascii" fails on FreeBSD on the master branch, whereas it pass before commit d19d8d5. This commit introduced the regression. Thanks for the bisection Nick! My change avoids modifying global configurable variables, whereas Py_DecodeLocale() and Py_EncodeLocale() depend on Py_UTF8Mode. |
|
The regression should now be fixed in 3.7 and master branches. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: