This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Concatenating bytes literal and f-string causes segmentation fault
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: Stone, eric.smith, ned.deily, python-dev, serhiy.storchaka, zach.ware
Priority: release blocker Keywords: patch

Created on 2016-11-07 17:46 by Stone, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
28633-0.diff eric.smith, 2016-11-07 20:53 review
28633-1.diff eric.smith, 2016-11-07 21:15 review
Messages (10)
msg280218 - (view) Author: Honor (Stone) Date: 2016-11-07 17:46
Hello,

Python version : 3.7.0a0
OS : Ubunt - Linux x 3.13.0-24-generic

Test Script:

>>> a="B\'\'F\'\'"
>>> eval(a)
Program received signal SIGSEGV, Segmentation fault.
0x0000000000531c5a in parsestrplus (n=0x7ffff7ee0b20, c=0x7fffffffd730) at
Python/ast.c:5150
5150            Py_DECREF(s);
(gdb) info reg
rax            0x0 0
rbx            0x0 0
rcx            0x7ffff7e9bab0 140737352678064
rdx            0x0 0
rsi            0x7ffff7e9ba88 140737352678024
rdi            0x7ffff7f74670 140737353565808
rbp            0x1 0x1
rsp            0x7fffffffd350 0x7fffffffd350
r8             0x0 0
r9             0x7fffffffd328 140737488343848
r10            0x7ffff7e9bab0 140737352678064
r11            0x7fffffffd2e0 140737488343776
r12            0x7ffff7ee0b20 140737352960800
r13            0x7fffffffd730 140737488344880
r14            0x0 0
r15            0x7ffff7f8557a 140737353635194
rip            0x531c5a 0x531c5a <ast_for_atom+3370>
eflags         0x10246 [ PF ZF IF RF ]
cs             0x33 51
ss             0x2b 43
ds             0x0 0
es             0x0 0
fs             0x0 0
gs             0x0 0

(gdb) bt
#0  0x0000000000531c5a in parsestrplus (n=0x7ffff7ee0b20, c=0x7fffffffd730)
at Python/ast.c:5150
#1  ast_for_atom (c=c@entry=0x7fffffffd730, n=0x7ffff7ee0b20) at
Python/ast.c:2110
#2  0x000000000053221a in ast_for_atom_expr (n=0x7ffff7ee0d78,
c=0x7fffffffd730) at Python/ast.c:2465
#3  ast_for_power (n=0x7ffff7ee0d50, c=0x7fffffffd730) at Python/ast.c:2502
#4  ast_for_expr (c=c@entry=0x7fffffffd730, n=0x7ffff7ee0d50) at
Python/ast.c:2690
#5  0x0000000000537446 in ast_for_testlist (n=0x7ffff7e8f0d0,
c=0x7fffffffd730) at Python/ast.c:2881
#6  PyAST_FromNodeObject (n=0x7ffff7ee0ad0, n@entry=0x7ffff7ee0af8,
flags=<optimized out>,
    filename=filename@entry=0x7ffff7e9be30, arena=arena@entry=0x7ffff7f751e0)
at Python/ast.c:815
#7  0x000000000042649f in PyParser_ASTFromStringObject
(arena=0x7ffff7f751e0, flags=<optimized out>, start=258,
    filename=0x7ffff7e9be30, s=0x7ffff7e9be30 "\003") at
Python/pythonrun.c:1124
#8  PyRun_StringFlags (str=str@entry=0x7ffff7e9bae0 "B''F''",
start=start@entry=258,
    globals=globals@entry=0x7ffff7f5d168, locals=locals@entry=0x7ffff7f5d168,
flags=flags@entry=0x7fffffffd840)
    at Python/pythonrun.c:902
#9  0x000000000053a9fe in builtin_eval_impl (module=<optimized out>,
locals=0x7ffff7f5d168, globals=0x7ffff7f5d168,
    source=0x7ffff7e9bab0) at Python/bltinmodule.c:875
#10 builtin_eval (module=<optimized out>, args=<optimized out>) at
Python/clinic/bltinmodule.c.h:243
#11 0x00000000004a7869 in _PyCFunction_FastCallDict (kwargs=0x0, nargs=1,
args=0x53a8b0 <builtin_eval>,
    func_obj=0x7ffff7fda990) at Objects/methodobject.c:234
#12 _PyCFunction_FastCallKeywords (func=func@entry=0x7ffff7fda990,
stack=stack@entry=0x7ffff7fa2ca8, nargs=1,
    kwnames=kwnames@entry=0x0) at Objects/methodobject.c:295
#13 0x000000000053c954 in call_function
(pp_stack=pp_stack@entry=0x7fffffffda50,
oparg=oparg@entry=1,
    kwnames=kwnames@entry=0x0) at Python/ceval.c:4793
#14 0x000000000054032c in _PyEval_EvalFrameDefault (f=<optimized out>,
throwflag=<optimized out>)
    at Python/ceval.c:3277
#15 0x000000000053c571 in PyEval_EvalFrameEx (throwflag=0,
f=0x7ffff7fa2b28) at Python/ceval.c:718
#16 _PyEval_EvalCodeWithName (_co=_co@entry=0x7ffff7ed7270,
globals=globals@entry=0x7ffff7f5d168,
    locals=locals@entry=0x7ffff7f5d168, args=args@entry=0x0,
argcount=argcount@entry=0, kwnames=kwnames@entry=0x0,
    kwargs=kwargs@entry=0x8, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2,
defs=defs@entry=0x0,
    defcount=defcount@entry=0, kwdefs=kwdefs@entry=0x0,
closure=closure@entry=0x0, name=name@entry=0x0,
    qualname=qualname@entry=0x0) at Python/ceval.c:4121
#17 0x000000000053d380 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0,
defcount=0, defs=0x0, kwcount=0, kws=0x0,
    argcount=0, args=0x0, locals=locals@entry=0x7ffff7f5d168,
globals=globals@entry=0x7ffff7f5d168,
    _co=_co@entry=0x7ffff7ed7270) at Python/ceval.c:4142
#18 PyEval_EvalCode (co=co@entry=0x7ffff7ed7270, globals=globals@entry
=0x7ffff7f5d168,
    locals=locals@entry=0x7ffff7f5d168) at Python/ceval.c:695
#19 0x0000000000427bc4 in run_mod (arena=0x7ffff7f75180,
flags=0x7fffffffdd40, locals=0x7ffff7f5d168,
    globals=0x7ffff7f5d168, filename=0x7ffff7f14ae8, mod=0x936ab0) at
Python/pythonrun.c:980
#20 PyRun_InteractiveOneObject (fp=fp@entry=0x7ffff74a9640
<_IO_2_1_stdin_>,
    filename=filename@entry=0x7ffff7f14ae8, flags=flags@entry=0x7fffffffdd40)
at Python/pythonrun.c:233
#21 0x0000000000427e8e in PyRun_InteractiveLoopFlags
(fp=fp@entry=0x7ffff74a9640
<_IO_2_1_stdin_>,
    filename_str=filename_str@entry=0x5d0f05 "<stdin>",
flags=flags@entry=0x7fffffffdd40)
at Python/pythonrun.c:112
#22 0x0000000000427f9c in PyRun_AnyFileExFlags (fp=0x7ffff74a9640
<_IO_2_1_stdin_>, filename=0x5d0f05 "<stdin>",
    closeit=0, flags=0x7fffffffdd40) at Python/pythonrun.c:74
#23 0x0000000000439b31 in run_file (p_cf=0x7fffffffdd40, filename=0x0,
fp=0x7ffff74a9640 <_IO_2_1_stdin_>)
    at Modules/main.c:319
#24 Py_Main (argc=argc@entry=1, argv=argv@entry=0x8fe010) at
Modules/main.c:779
#25 0x000000000041d964 in main (argc=1, argv=<optimized out>) at
./Programs/python.c:69
msg280222 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-11-07 18:05
Reproduced on macOS:

$ ./python.exe
Python 3.6.0b4+ (3.6:b26c8104e54f, Nov  7 2016, 12:01:37)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type help, copyright, credits or license for more information.
>>> B''F''
Segmentation fault: 11

Looks like `f''b''` is handled correctly, but `b''f''` is not.
msg280224 - (view) Author: Honor (Stone) Date: 2016-11-07 18:15
Why not?

I have tested it.
Different payload : '%%-'%B'4--'F''
Again crashed.
Can you try?

On Mon, Nov 7, 2016 at 9:14 PM, Serhiy Storchaka <report@bugs.python.org>
wrote:

>
> Changes by Serhiy Storchaka <storchaka+cpython@gmail.com>:
>
>
> ----------
> nosy: +serhiy.storchaka
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28633>
> _______________________________________
>
msg280225 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-11-07 19:08
Works:

>>> f'' b'' 
  File "<stdin>", line 1
SyntaxError: cannot mix bytes and nonbytes literals

Fails:

>>> b'' f''
Segmentation fault
$

Regular strings work:
>>> '' b''
  File "<stdin>", line 1
SyntaxError: cannot mix bytes and nonbytes literals
>>> b'' ''
  File "<stdin>", line 1
SyntaxError: cannot mix bytes and nonbytes literals
>>>
msg280228 - (view) Author: Honor (Stone) Date: 2016-11-07 19:45
Another:

>>> 9,'''Ðؚ%''(r'''%b''''"r'''''
  File "<stdin>", line 1
SyntaxError: cannot mix bytes and nonbytes literals
>>> 9,'B''Ðؚ%''(r'''%b''''"r'''
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: incomplete format key
>>> 9,'B''Ðؚ%''(r'''%b''''"r'''F
  File "<stdin>", line 1
    9,'B''Ðؚ%''(r'''%b''''"r'''F
                               ^
SyntaxError: invalid syntax
>>> 9,'B''Ðؚ%''(r'''%b''''"r'''F'
  File "<stdin>", line 1
    9,'B''Ðؚ%''(r'''%b''''"r'''F'
                                 ^
SyntaxError: EOL while scanning string literal
>>>

On Mon, Nov 7, 2016 at 10:08 PM, Eric V. Smith <report@bugs.python.org>
wrote:

>
> Eric V. Smith added the comment:
>
> Works:
>
> >>> f'' b''
>   File "<stdin>", line 1
> SyntaxError: cannot mix bytes and nonbytes literals
>
> Fails:
>
> >>> b'' f''
> Segmentation fault
> $
>
> Regular strings work:
> >>> '' b''
>   File "<stdin>", line 1
> SyntaxError: cannot mix bytes and nonbytes literals
> >>> b'' ''
>   File "<stdin>", line 1
> SyntaxError: cannot mix bytes and nonbytes literals
> >>>
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28633>
> _______________________________________
>
msg280235 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-11-07 20:34
The ones in msg280228 give correct error messages.
msg280239 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-11-07 20:53
It's a decref of a NULL pointer. Patch with test is attached.
msg280243 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-07 21:10
LGTM. I added comments about other asserts. Not related, but would be nice to fix them too while we are here.
msg280245 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-11-07 21:15
Thanks, Serhiy. I updated the patch.
msg280255 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-11-07 22:58
New changeset 31543f7cbdf4 by Eric V. Smith in branch '3.6':
Fixed issue #28633: segfault when concatenating bytes literal and f-string.
https://hg.python.org/cpython/rev/31543f7cbdf4
History
Date User Action Args
2022-04-11 14:58:39adminsetgithub: 72819
2016-11-07 23:00:30eric.smithsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2016-11-07 22:58:11python-devsetnosy: + python-dev
messages: + msg280255
2016-11-07 21:15:34eric.smithsetfiles: + 28633-1.diff

messages: + msg280245
2016-11-07 21:10:32serhiy.storchakasetmessages: + msg280243
2016-11-07 20:53:34eric.smithsetfiles: + 28633-0.diff
keywords: + patch
messages: + msg280239

stage: needs patch -> patch review
2016-11-07 20:34:23eric.smithsetmessages: + msg280235
2016-11-07 19:45:27Stonesetmessages: + msg280228
2016-11-07 19:08:17eric.smithsetmessages: + msg280225
2016-11-07 18:42:59eric.smithsetassignee: eric.smith
2016-11-07 18:21:56serhiy.storchakasettitle: eval() Function - Segmentation Fault -> Concatenating bytes literal and f-string causes segmentation fault
2016-11-07 18:15:37Stonesetmessages: + msg280224
2016-11-07 18:14:47serhiy.storchakasetnosy: + serhiy.storchaka
2016-11-07 18:05:19zach.waresetpriority: normal -> release blocker

type: crash
components: + Interpreter Core
versions: + Python 3.6, Python 3.7
nosy: + eric.smith, ned.deily, zach.ware

messages: + msg280222
stage: needs patch
2016-11-07 17:46:08Stonecreate