classification
Title: python3.6 -Werror -c '"\c"' fails with an assertion error
Type: crash Stage: patch review
Components: Interpreter Core Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: abarry, eric.smith, ned.deily, python-dev, serhiy.storchaka, vstinner
Priority: release blocker Keywords: patch

Created on 2016-11-14 16:53 by vstinner, last changed 2016-11-15 08:14 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
ast.patch vstinner, 2016-11-14 17:04 review
test_eval_str_invalid_escape_error.patch serhiy.storchaka, 2016-11-14 18:53 review
Messages (7)
msg280793 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-11-14 16:53
The issue #28128 (changeset 259745f9a1e4) introduced a DeprecationWarning on invalid Unicode sequence like "\c". When Python is run with -Werror, Python crashs with an assertion error:

$ ./python -Werror -c '"\c"'
python: Objects/abstract.c:2232: PyObject_Call: Assertion `!PyErr_Occurred()' failed.
Aborted (core dumped)

Gdb traceback:

(gdb) where
#0  0x00007ffff711f6f5 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff71212fa in __GI_abort () at abort.c:89
#2  0x00007ffff7117f97 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x69b577 "!PyErr_Occurred()", file=file@entry=0x69a922 "Objects/abstract.c", line=line@entry=2232, 
    function=function@entry=0x69b9c8 <__PRETTY_FUNCTION__.11084> "PyObject_Call") at assert.c:92
#3  0x00007ffff7118042 in __GI___assert_fail (assertion=0x69b577 "!PyErr_Occurred()", file=0x69a922 "Objects/abstract.c", line=2232, function=0x69b9c8 <__PRETTY_FUNCTION__.11084> "PyObject_Call")
    at assert.c:101
#4  0x000000000045afe8 in PyObject_Call (func=<type at remote 0x938a80>, args=(2, 'No such file or directory', '<stdin>'), kwargs=0x0) at Objects/abstract.c:2232
#5  0x00000000005e77de in PyErr_SetFromErrnoWithFilenameObjects (exc=<type at remote 0x938a80>, filenameObject='<stdin>', filenameObject2=0x0) at Python/errors.c:559
#6  0x00000000005e7631 in PyErr_SetFromErrnoWithFilenameObject (exc=<type at remote 0x938a80>, filenameObject='<stdin>') at Python/errors.c:471
#7  0x000000000043e839 in _Py_fopen_obj (path='<stdin>', mode=0x6c86e7 "rb") at Python/fileutils.c:1138
#8  0x00000000005e8e54 in PyErr_ProgramTextObject (filename='<stdin>', lineno=1) at Python/errors.c:1179
#9  0x0000000000598e90 in ast_error (c=0x7fffffffd8b0, n=0x7ffff02d31f0, errmsg=0x7ffff02c1a80 "invalid escape sequence \\c") at Python/ast.c:673
#10 0x00000000005a3ae8 in warn_invalid_escape_sequence (c=0x7fffffffd8b0, n=0x7ffff02d31f0, first_invalid_escape_char=99 'c') at Python/ast.c:4134
#11 0x00000000005a4123 in decode_unicode_with_escapes (c=0x7fffffffd8b0, n=0x7ffff02d31f0, s=0x7ffff03157f0 "\\c\313\313\313\313\313\313\313\313\313", <incomplete sequence \313>, len=2)
    at Python/ast.c:4202
#12 0x00000000005a63b8 in parsestr (c=0x7fffffffd8b0, n=0x7ffff02d31f0, bytesmode=0x7fffffffd29c, rawmode=0x7fffffffd298, result=0x7fffffffd290, fstr=0x7fffffffd288, fstrlen=0x7fffffffd280)
    at Python/ast.c:5114
#13 0x00000000005a64cb in parsestrplus (c=0x7fffffffd8b0, n=0x7ffff02d31a8) at Python/ast.c:5145
#14 0x000000000059d17b in ast_for_atom (c=0x7fffffffd8b0, n=0x7ffff02d31a8) at Python/ast.c:2112
#15 0x000000000059e4d9 in ast_for_atom_expr (c=0x7fffffffd8b0, n=0x7ffff02d3160) at Python/ast.c:2467
#16 0x000000000059e649 in ast_for_power (c=0x7fffffffd8b0, n=0x7ffff02d3118) at Python/ast.c:2504
#17 0x000000000059ef95 in ast_for_expr (c=0x7fffffffd8b0, n=0x7ffff02d3118) at Python/ast.c:2692
#18 0x000000000059f8d2 in ast_for_testlist (c=0x7fffffffd8b0, n=0x7ffff0351d30) at Python/ast.c:2883
#19 0x000000000059f992 in ast_for_expr_stmt (c=0x7fffffffd8b0, n=0x7ffff0351ce8) at Python/ast.c:2906
#20 0x00000000005a3475 in ast_for_stmt (c=0x7fffffffd8b0, n=0x7ffff0351ce8) at Python/ast.c:3983
#21 0x00000000005997e8 in PyAST_FromNodeObject (n=0x7ffff0351c10, flags=0x7fffffffdcf0, filename='<stdin>', arena=0x7ffff02ec040) at Python/ast.c:839
#22 0x000000000042a146 in PyParser_ASTFromFileObject (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename='<stdin>', enc=0x7ffff7e574f8 "UTF-8", start=256, ps1=0x7ffff02cdd18 ">>> ", 
    ps2=0x7ffff02cdde8 "... ", flags=0x7fffffffdcf0, errcode=0x7fffffffda4c, arena=0x7ffff02ec040) at Python/pythonrun.c:1169
#23 0x0000000000426b00 in PyRun_InteractiveOneObject (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename='<stdin>', flags=0x7fffffffdcf0) at Python/pythonrun.c:212
#24 0x0000000000426698 in PyRun_InteractiveLoopFlags (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename_str=0x697276 "<stdin>", flags=0x7fffffffdcf0) at Python/pythonrun.c:112
#25 0x0000000000426491 in PyRun_AnyFileExFlags (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename=0x697276 "<stdin>", closeit=0, flags=0x7fffffffdcf0) at Python/pythonrun.c:74
#26 0x00000000004421cd in run_file (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename=0x0, p_cf=0x7fffffffdcf0) at Modules/main.c:319
#27 0x0000000000443086 in Py_Main (argc=2, argv=0x9d2010) at Modules/main.c:779
#28 0x000000000041d2c9 in main (argc=2, argv=0x7fffffffdf28) at ./Programs/python.c:69
msg280794 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-11-14 16:55
By the way, an unit test is missing since the bug was not catched by the test suite!
msg280796 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-11-14 17:04
Attached ast.patch fixes the issue: replace the DeprecationWarning with a SyntaxError.

Is it the expected behaviour? Is it worth it to chain the two exceptions?
msg280797 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-11-14 17:05
> By the way, an unit test is missing since the bug was not catched by the test suite!

Lib/test/test_string_literals.py contains an unit test but it changes the warnings filter, so -Werror option is ignored.
msg280805 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-14 18:53
This is my fault. Proposed patch adds tests for the "error" action.
msg280826 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-11-15 08:13
New changeset aa52ea2a7731 by Victor Stinner in branch '3.6':
Fix warn_invalid_escape_sequence()
https://hg.python.org/cpython/rev/aa52ea2a7731
msg280827 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-11-15 08:14
Thanks Serhiy for your unit tests! I pushed with patch with your tests.
History
Date User Action Args
2016-11-15 08:14:16vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg280827
2016-11-15 08:13:07python-devsetnosy: + python-dev
messages: + msg280826
2016-11-14 18:53:27serhiy.storchakasetfiles: + test_eval_str_invalid_escape_error.patch

type: crash
components: + Interpreter Core

nosy: + serhiy.storchaka
messages: + msg280805
stage: patch review
2016-11-14 17:05:57vstinnersetmessages: + msg280797
2016-11-14 17:04:53vstinnersetfiles: + ast.patch
keywords: + patch
messages: + msg280796
2016-11-14 16:55:40vstinnersetmessages: + msg280794
2016-11-14 16:53:06vstinnercreate