This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Calling "functions" used to implement generators/comps easily cause crash
Type: crash Stage:
Components: Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jeffrey.Kintscher, bup, iritkatriel, rhettinger
Priority: normal Keywords:

Created on 2019-05-18 13:53 by bup, last changed 2022-04-11 14:59 by admin.

Messages (5)
msg342797 - (view) Author: Dan Snider (bup) * Date: 2019-05-18 13:53
As far as I know, generators, set comprehensions, list comprehensions, and dict comprehensions, (along with their asynchronous variants) are implemented by first calling the GET_(A)ITER opcode and then building and calling a function that acepts the resulting iterator as its sole argument.

Assigning the code object used to make that function (or using it in the types.FunctionType constructor) and then calling it with a non-iterator argument will obviously cause a crash since the FOR_ITER opcode rightly expects that it will never have to deal with non-iterators and calls tp_iternext without checking if it exists.

The 4-liner demonstrates the crash:

if 1:
  fn = lambda: None
  gi = (i for i in ())
  fn.__code__ = gi.gi_code
  [*fn("abc")]
msg346807 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-06-28 08:02
What is the preferred behavior? Raise an exception of some kind?
msg346808 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-06-28 08:13
Specifically, the crash occurs in Python/ceval.c function _PyEval_EvalFrameDefault() in the TARGET(FOR_ITER) case at line 3198 (master branch):

PyObject *next = (*iter->ob_type->tp_iternext)(iter);

It segfaults because tp_iternext is NULL.
msg346809 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-06-28 08:15
ISTM this report is close to that covered by the traditional stance that word code hacks aren't protected from crashes because trying to do can be complicated, can be expensive, and isn't always possible.  

I'm not really sure where this falls on the grey scale with "normal, everyday python shouldn't crash" on one end and "with ctypes or code hacks, all bets are off" on the other end.  

Offhand, the posted snippet seems closer to "messing with the internals" but perhaps small cheap check can be added when the __code__ variable is updated.
msg402171 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-09-19 21:54
Reproduced on 3.11.
History
Date User Action Args
2022-04-11 14:59:15adminsetgithub: 81137
2021-09-19 21:54:42iritkatrielsetnosy: + iritkatriel

messages: + msg402171
versions: + Python 3.10, Python 3.11, - Python 3.6, Python 3.7, Python 3.8
2019-06-28 08:15:22rhettingersetnosy: + rhettinger
messages: + msg346809
2019-06-28 08:13:01Jeffrey.Kintschersetmessages: + msg346808
2019-06-28 08:02:20Jeffrey.Kintschersetmessages: + msg346807
2019-06-16 23:55:01huntertursetversions: + Python 3.9
2019-05-28 08:44:51Jeffrey.Kintschersetnosy: + Jeffrey.Kintscher
2019-05-18 13:53:21bupcreate