This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: closure with too few cells segfaults
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.3
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, daniel.urban, eric.snow
Priority: normal Keywords:

Created on 2011-03-24 07:00 by eric.snow, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
unnamed eric.snow, 2011-03-24 21:47
unnamed eric.snow, 2011-03-24 21:51
Messages (6)
msg131961 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2011-03-24 07:00
While perhaps esoteric, it looks like exec'ing a code object that has freevars, using a closure that has too few cells causes a segfault.  I believe the problem is around line 3276 of ceval.c at the PyTuple_GET_ITEM call:

    if (PyTuple_GET_SIZE(co->co_freevars)) {
        int i;
        for (i = 0; i < PyTuple_GET_SIZE(co->co_freevars); ++i) {
>>>         PyObject *o = PyTuple_GET_ITEM(closure, i);
            Py_INCREF(o);
            freevars[PyTuple_GET_SIZE(co->co_cellvars) + i] = o;
        }
    }

I only bring this up because I am toying around with exposing a wrapper around PyEval_EvalCodeEx that is a more fully featured version of exec.  Here is an example of code where I ran into the problem:

def outer():
    x = 5
    y = 6
    def f(): return x,y
    z = 7
    def g(): return z
    exec_closure(f.__code__, closure=g.__closure__)

Incidently, it looks there isn't any check to see if len(closure) > len(freevars), which I would expect to be disallowed.  However, I understand that there hasn't really been any point to worry about it due to the current usage of PyEval_EvalCodeEx.  

If the above two constraints are appropriate I would love to see them added in with something like the following:

    if (closure == NULL)
        &closure = PyTuple_New(0);
    if (!PyTuple_Check(closure)) {
        PyErr_Format(PyExc_TypeError,
                     "closure must be a tuple");
        goto fail;        
    }
    Py_ssize_t nfreevars = PyTuple_GET_SIZE(co->co_freevars);
    Py_ssize_t ncells = PyTuple_GET_SIZE(closure);
    if (nfreevars != ncells) {
        PyErr_Format(PyExc_SystemError,
                     "Expected %s cells, received %s", 
                     nfreevars, ncells);
        goto fail;        
    }
    if (nfreevars) {
        int i;
        for (i = 0; i < PyTuple_GET_SIZE(co->co_freevars); ++i) {
            PyObject *o = PyTuple_GET_ITEM(closure, i);
            Py_INCREF(o);
            freevars[PyTuple_GET_SIZE(co->co_cellvars) + i] = o;
        }
    }

Alternately, I could just add some validation into exec_closure, if it's not worth bothering in ceval.c.
msg132035 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-03-24 21:38
And the source of exec_closure is what exactly?
msg132038 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2011-03-24 21:47
I'm hoping to "release" it this week.  Nick helped be a bunch with it during
pycon sprints.  I basically took call_function in ceval.c and got it working
for arbitrary code blocks and exposed it as exec_closure to look like exec.

On Thu, Mar 24, 2011 at 3:38 PM, Benjamin Peterson
<report@bugs.python.org>wrote:

>
> Benjamin Peterson <benjamin@python.org> added the comment:
>
> And the source of exec_closure is what exactly?
>
> ----------
> nosy: +benjamin.peterson
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue11660>
> _______________________________________
>
msg132039 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-03-24 21:48
2011/3/24 Eric Snow <report@bugs.python.org>:
>
> Eric Snow <ericsnowcurrently@gmail.com> added the comment:
>
> I'm hoping to "release" it this week.  Nick helped be a bunch with it during
> pycon sprints.  I basically took call_function in ceval.c and got it working
> for arbitrary code blocks and exposed it as exec_closure to look like exec.

Is it a Python or C function?
msg132040 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2011-03-24 21:51
It's a C function that for now I have in an extension module.  If it turns
out to be useful I am going to try to get it put into the builtins, but I
don't want to get ahead of myself.
msg132042 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-03-24 21:53
Then the checking should be in your function.
History
Date User Action Args
2022-04-11 14:57:15adminsetgithub: 55869
2011-03-24 21:53:13benjamin.petersonsetstatus: open -> closed
resolution: wont fix
messages: + msg132042
2011-03-24 21:51:14eric.snowsetfiles: + unnamed

messages: + msg132040
2011-03-24 21:48:22benjamin.petersonsetmessages: + msg132039
2011-03-24 21:47:12eric.snowsetfiles: + unnamed

messages: + msg132038
2011-03-24 21:46:37daniel.urbansetnosy: + daniel.urban
2011-03-24 21:38:50benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg132035
2011-03-24 07:00:29eric.snowcreate