This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: FunctionType.__new__ can generate functions that immediately crash
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.4, Python 3.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Dennis Sweeney, bup, rhettinger, xtreak
Priority: normal Keywords:

Created on 2018-07-23 03:57 by bup, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg322174 - (view) Author: Dan Snider (bup) * Date: 2018-07-23 03:57
The following program crashes on 3.3.1, 3.4.2, 3.5.2, and 3.6.1 because despite having a non-empty co_cellvars slot due to the generator object, the NOFREE flag was set. 3.7 isn't immune to some bad behavior here, either. 

While I only have access to 3.7.03b right now, I noted the assertion at the bottom failing because it appears CodeType.__new__ is silently stripping the NOFREE flag out, which is why it's highlighted as well.

Since the interpreter doesn't actually use FunctionType.__new__ it shouldn't hurt performance too much to add a check for when NOFREE is set that co_cellvars and co_freevars are indeed empty. Anyway, the code:

from functools import reduce
from itertools import repeat
from operator import or_
from types import FunctionType, CodeType

OPTIMIZED, NEWLOCALS, NOFREE = 1, 2, 64
FLAGS = [OPTIMIZED, NEWLOCALS, NOFREE]
fields=('argcount kwonlyargcount nlocals stacksize flags code consts '
'names varnames filename name firstlineno lnotab freevars cellvars').split()

def edit_code(code, *args, **kws):
    """Construct a new code object using `code` as a template"""

    params = []
    atrs = ('co_%s'%a for a in fields)
    kwds = map(kws.pop, fields, repeat(kws))
    for arg, kwv, k in zip(args, kwds, atrs):
        if kwv is not kws:
            raise TypeError("edit_code() got multiple parameters for %r"%k)
        params.append(arg)
    for kwv, atr in zip(kwds, atrs):
        params.append(kwv if kwv is not kws else getattr(code, atr))
    if kws:
        k, v = kws.popitem()
        raise TypeError("edit_code() got unexpected keyword argument %r"%k)
    return CodeType(*params)
        
def get_co_flags(flags):
    if isinstance(flags, FunctionType):
        flags = flags.__code__.co_flags
    elif isinstance(flags, CodeType):
        flags = flags.co_flags
    return reduce(or_, (i for i in FLAGS if flags & i))

if __name__ == '__main__':
    co = get_co_flags.__code__
    ns = get_co_flags.__globals__
    flags = co.co_flags
    assert flags == OPTIMIZED|NEWLOCALS
    assert NOFREE == 64
    a = FunctionType(edit_code(co, flags=flags), ns)
    b = FunctionType(edit_code(co, flags=flags|NOFREE), ns)
    # this assertion fails on 3.7.0b3
    assert b.__code__.co_flags == OPTIMIZED|NEWLOCALS|NOFREE
    print('calling a...')
    a(get_co_flags)
    t = input("Blow up the universe? y/n : ")
    if t != 'n':
        b(get_co_flags)
msg322338 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-25 06:48
The assertion also fails on master. I did some manual git blame work. The assertion passes with commit 7324b5ce8e7c031a0a3832a6a8d7c639111ae0ff. It fails with the next commit 078f1814f1a4413a2a0fdb8cf4490ee0fc98ef34 (https://bugs.python.org/issue32176) that has some changes with respect to CO_NOFREE. The change was released with 3.7.0a3 as noted in the comment (https://bugs.python.org/issue32176#msg307512) and hence the assertion should fail on 3.7.0b3 also.

# commit 7324b5c

➜  cpython git:(7324b5c) ✗ ./python
Python 3.7.0a2+ (v3.7.0a2-334-g7324b5c:7324b5c, Jul 25 2018, 06:34:52)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
➜  cpython git:(7324b5c) ✗ ./python bpo34192.py
calling a...
Blow up the universe? y/n : n

# commit 078f181

➜  cpython git:(078f181) ✗ ./python
Python 3.7.0a2+ (v3.7.0a2-335-g078f181:078f181, Jul 25 2018, 06:36:32)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
➜  cpython git:(078f181) ✗ ./python bpo34192.py
Traceback (most recent call last):
  File "bpo34192.py", line 44, in <module>
    assert b.__code__.co_flags == OPTIMIZED|NEWLOCALS|NOFREE
AssertionError


Thanks
msg404220 - (view) Author: Dennis Sweeney (Dennis Sweeney) * (Python committer) Date: 2021-10-18 20:53
From https://github.com/python/cpython/blob/main/Lib/test/crashers/bogus_code_obj.py :


"""
Broken bytecode objects can easily crash the interpreter.

This is not going to be fixed.  It is generally agreed that there is no
point in writing a bytecode verifier and putting it in CPython just for
this.  Moreover, a verifier is bound to accept only a subset of all safe
bytecodes, so it could lead to unnecessary breakage.
"""

Since this is messing with implementation details of code objects, I'll close this as "won't fix".
History
Date User Action Args
2022-04-11 14:59:03adminsetgithub: 78373
2021-10-18 20:53:29Dennis Sweeneysetstatus: open -> closed

nosy: + Dennis Sweeney
messages: + msg404220

resolution: wont fix
stage: resolved
2018-07-25 06:48:34xtreaksetmessages: + msg322338
2018-07-24 17:09:29xtreaksetnosy: + xtreak
2018-07-23 06:20:01rhettingersetnosy: + rhettinger
2018-07-23 03:57:25bupcreate