classification
Title: frame.f_locals causes segfault on Python >=3.4.1
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, msmhrt, pitrou, python-dev, skrah
Priority: normal Keywords: patch

Created on 2014-07-01 13:40 by msmhrt, last changed 2014-07-05 00:31 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
f_locals_clear.patch pitrou, 2014-07-02 22:59 review
Messages (7)
msg222029 - (view) Author: Masami HIRATA (msmhrt) Date: 2014-07-01 13:40
It seems that frame.f_locals causes segfault on Python >=3.4.1

$ uname -a
Linux ashrose 3.2.0-61-generic #93-Ubuntu SMP Fri May 2 21:31:50 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
$ ls
test1.py
$ cat test1.py
import unittest

class TestCallable(unittest.TestCase):
    def test_callable(self):
        try:
            with self.assertRaises(IndexError):
                def raise_error(): raise TypeError
                (lambda: raise_error())()
        except TypeError as exception:
            exception.__traceback__.tb_next.tb_frame.f_locals
$ python3.4.1 -m unittest
Segmentation fault
$ python3.4.0 -m unittest
.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK
$ python3.3.5 -m unittest
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
$ gdb /home/python/local/3.4.1mgs/bin/python3.4.1

...

(gdb) run -m unittest
Starting program: /home/python/local/3.4.1mgs/bin/python3.4.1 -m unittest
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
map_to_dict (deref=1, values=0x7ffff5e97510, dict=0x7ffff5e8f588, nmap=<optimized out>, map=0x7ffff60e0278)
    at Objects/frameobject.c:791
791	            value = PyCell_GET(value);
(gdb) bt
#0  map_to_dict (deref=1, values=0x7ffff5e97510, dict=0x7ffff5e8f588, nmap=<optimized out>, 
    map=0x7ffff60e0278) at Objects/frameobject.c:791
#1  PyFrame_FastToLocalsWithError (f=0x7ffff5e97398) at Objects/frameobject.c:915
#2  0x00007ffff79beaa9 in frame_getlocals (f=0x7ffff5e97398, closure=<optimized out>)
    at Objects/frameobject.c:24
#3  0x00007ffff79deff3 in _PyObject_GenericGetAttrWithDict (obj=0x7ffff5e97398, name=0x7ffff7fcdbb0, dict=0x0)
    at Objects/object.c:1043
#4  0x00007ffff7a57891 in PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2411
#5  0x00007ffff7a5df3c in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>, 
    locals=<optimized out>, args=<optimized out>, argcount=1, kws=0x790f48, kwcount=0, defs=0x0, defcount=0, 
    kwdefs=0x0, closure=0x0) at Python/ceval.c:3578
#6  0x00007ffff7a5ca61 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, 
    pp_stack=0x7fffffffbbf0, func=0x7ffff6115158) at Python/ceval.c:4334
#7  call_function (oparg=<optimized out>, pp_stack=0x7fffffffbbf0) at Python/ceval.c:4252
#8  PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2829
#9  0x00007ffff7a5df3c in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>, 
    locals=<optimized out>, args=<optimized out>, argcount=2, kws=0x7ffff7fad060, kwcount=0, 
    defs=0x7ffff617cd48, defcount=1, kwdefs=0x0, closure=0x0) at Python/ceval.c:3578
#10 0x00007ffff79bf28c in function_call (func=0x7ffff6192400, arg=0x7ffff6101c08, kw=0x7ffff5e8f248)
    at Objects/funcobject.c:632
#11 0x00007ffff799300e in PyObject_Call (func=0x7ffff6192400, arg=<optimized out>, kw=<optimized out>)
    at Objects/abstract.c:2067
#12 0x00007ffff7a56871 in ext_do_call (nk=0, na=1, flags=<optimized out>, pp_stack=0x7fffffffbec0, 
    func=0x7ffff6192400) at Python/ceval.c:4551
#13 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2869
---Type <return> to continue, or q <return> to quit---
msg222045 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-07-01 15:16
6ab3193e890e exposes the issue.
msg222061 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-07-02 00:16
Following patch seems to fix it, but I have to cook a proper test:

diff --git a/Objects/frameobject.c b/Objects/frameobject.c
--- a/Objects/frameobject.c
+++ b/Objects/frameobject.c
@@ -786,7 +786,7 @@ map_to_dict(PyObject *map, Py_ssize_t nm
         PyObject *key = PyTuple_GET_ITEM(map, j);
         PyObject *value = values[j];
         assert(PyUnicode_Check(key));
-        if (deref) {
+        if (deref && value != NULL) {
             assert(PyCell_Check(value));
             value = PyCell_GET(value);
         }
msg222141 - (view) Author: Masami HIRATA (msmhrt) Date: 2014-07-02 22:52
Thanks, pitrou.

Your patch seems ok to me.

$ python3.4.1_with_patch -m unittest
.
----------------------------------------------------------------------
Ran 1 test in 0.001s

It seems that this issue is same as https://bitbucket.org/hpk42/pytest/issue/528/test-causes-segfault .
msg222142 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-07-02 22:59
Here is a patch with tests.
msg222324 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-05 00:30
New changeset 758468cdf72c by Antoine Pitrou in branch '3.4':
Issue #21897: Fix a crash with the f_locals attribute with closure variables when frame.clear() has been called.
http://hg.python.org/cpython/rev/758468cdf72c

New changeset bd6515070f9c by Antoine Pitrou in branch 'default':
Issue #21897: Fix a crash with the f_locals attribute with closure variables when frame.clear() has been called.
http://hg.python.org/cpython/rev/bd6515070f9c
msg222325 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-07-05 00:31
Patch committed. Thank you for reporting this issue!
History
Date User Action Args
2014-07-05 00:31:25pitrousetstatus: open -> closed
resolution: fixed
messages: + msg222325

stage: patch review -> resolved
2014-07-05 00:30:40python-devsetnosy: + python-dev
messages: + msg222324
2014-07-02 22:59:58pitrousetfiles: + f_locals_clear.patch

versions: + Python 3.5
keywords: + patch
nosy: + benjamin.peterson

messages: + msg222142
stage: patch review
2014-07-02 22:52:52msmhrtsetmessages: + msg222141
2014-07-02 00:16:16pitrousetmessages: + msg222061
2014-07-01 15:16:35skrahsetnosy: + skrah, pitrou
messages: + msg222045
2014-07-01 13:40:08msmhrtcreate