classification
Title: Segfault with StopIteration
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: gregory.p.smith, kchen
Priority: normal Keywords:

Created on 2019-10-24 01:50 by kchen, last changed 2019-10-25 21:22 by kchen. This issue is now closed.

Messages (3)
msg355274 - (view) Author: Kevin Chen (kchen) Date: 2019-10-24 01:50
Unfortunately, I don't currently have a simple test case, but after migrating an app from python 3.6 to python 3.7, python frequently (but not always) segfaults when running tests for my app in code related to StopIteration (perhaps somehow related to the implementation of PEP 479?).

I'm running a self-built (from python.org sources) unpatched build of Python 3.7.5 on Ubuntu Xenial.

(The output below is edited to redact some path names.)

Core was generated by `./python'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000000000044cc40 in PyException_GetContext (self=self@entry=0x441cbe0)
    at Objects/exceptions.c:338
338	    Py_XINCREF(context);

(gdb) py-bt
Traceback (most recent call first):
  File "lib/requests/models.py", line 750, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "lib/requests/models.py", line 828, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "lib/requests/sessions.py", line 686, in send
    r.content
  File "lib/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "lib/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
[trace continues with other files specific to my app]

(gdb) bt
#0  0x000000000044cc40 in PyException_GetContext (
    self=self@entry=<unknown at remote 0x441cbe0>) at Objects/exceptions.c:338
#1  0x000000000050b76d in PyErr_SetObject (
    exception=<type at remote 0x8b7c00>, value=StopIteration())
    at Python/errors.c:126
#2  0x000000000050ba17 in PyErr_SetNone (exception=<optimized out>)
    at Python/errors.c:163
#3  0x00000000005d254d in gen_send_ex (gen=0x4351b00, arg=<optimized out>, 
    exc=<optimized out>, closing=<optimized out>) at Objects/genobject.c:241
#4  0x0000000000429406 in _PyEval_EvalFrameDefault (f=<optimized out>, 
    throwflag=<optimized out>) at Python/ceval.c:2809
#5  0x00000000005d20cb in gen_send_ex (gen=0x457c1b0, arg=<optimized out>, 
    exc=<optimized out>, closing=<optimized out>) at Objects/genobject.c:221
#6  0x0000000000459679 in list_extend (self=self@entry=0x445a590, 
    iterable=iterable@entry=<generator at remote 0x457c1b0>)
    at Objects/listobject.c:897
#7  0x000000000045de55 in _PyList_Extend (self=self@entry=0x445a590, 
    iterable=iterable@entry=<generator at remote 0x457c1b0>)
    at Objects/listobject.c:937
#8  0x00000000005b76fc in PySequence_List (
    v=v@entry=<generator at remote 0x457c1b0>) at Objects/abstract.c:1905
#9  0x00000000005b867d in PySequence_Fast (v=<optimized out>, 
    m=m@entry=0x603715 "can only join an iterable") at Objects/abstract.c:1935
#10 0x0000000000438e02 in stringlib_bytes_join (sep=b'', 
    iterable=<optimized out>) at Objects/stringlib/join.h:22
#11 0x0000000000445b53 in _PyMethodDef_RawFastCallKeywords (
    method=0x8b0cc0 <bytes_methods+608>, self=self@entry=b'', 
    args=args@entry=0x4322f20, nargs=nargs@entry=1, kwnames=kwnames@entry=0x0)
    at Objects/call.c:648
#12 0x00000000005cfabf in _PyMethodDescr_FastCallKeywords (
    descrobj=<method_descriptor at remote 0x16f99f0>, 
    args=args@entry=0x4322f18, nargs=nargs@entry=2, kwnames=kwnames@entry=0x0)
    at Objects/descrobject.c:288
#13 0x000000000042df85 in call_function (kwnames=0x0, oparg=2, 
    pp_stack=<synthetic pointer>) at Python/ceval.c:4593
#14 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:3110
#15 0x00000000004211c5 in function_code_fastcall (co=co@entry=0x3597c30, 
    args=<optimized out>, args@entry=0x3ce1f08, nargs=nargs@entry=1, 
    globals=globals@entry={'__name__': 'requests.models', '__doc__': '\nrequests.models\n~~~~~~~~~~~~~~~\n\nThis module contains the primary objects that power Requests.\n', '__package__': 'requests', '__loader__': <SourceFileLoader(name='requests.models', path='lib/requests/models.py') at remote 0x357cf70>, '__spec__': <ModuleSpec(name='requests.models', loader=<...>, origin='lib/requests/models.py', loader_state=None, submodule_search_locations=None, _set_fileattr=True, _cached='lib/requests/__pycache__/models.cpython-37.pyc', _initializing=False) at remote 0x354cd70>, '__file__': 'lib/requests/models.py', '__cached__': 'lib/requests/__pycache__/models.cpython-37.pyc', '__builtins__': {'__name__': 'builtins', '__doc__': "Built-in functions, exceptions, and other objects.\n\nNoteworthy: N...(truncated)) at Objects/call.c:283
#16 0x0000000000445180 in _PyFunction_FastCallDict (
    func=<function at remote 0x35b41c0>, args=0x3ce1f08, nargs=1, kwargs=0x0)
    at Objects/call.c:322
[remainder of trace redacted]

Sometimes the C backtrace is slightly different, but very similar:

Core was generated by `./python'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  PyErr_SetObject (exception=0x8b7c00 <_PyExc_StopIteration>, value=0x0)
    at Python/errors.c:101
101	        Py_INCREF(exc_value);

(gdb) bt
#0  PyErr_SetObject (exception=<type at remote 0x8b7c00>, value=0x0)
    at Python/errors.c:101
#1  0x000000000050ba17 in PyErr_SetNone (exception=<optimized out>)
    at Python/errors.c:163
#2  0x00000000005d254d in gen_send_ex (gen=0x3c71b10, arg=<optimized out>, 
    exc=<optimized out>, closing=<optimized out>) at Objects/genobject.c:241
[rest of trace not listed, since it appears to be the same as the previous one]

When running my app's tests, I've also seen one occurrence (possibly related to these crashes) when Python aborted because a reference count was -6.
msg355279 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-10-24 06:15
There isn't really much we can do without a reasonable repeatable way to reproduce the problem.
msg355387 - (view) Author: Kevin Chen (kchen) Date: 2019-10-25 21:22
Sorry for the noise.  I upgraded my gevent (1.3.5 to 1.4.0) and greenlet (0.4.13 to 0.4.15) libraries, and that appears to have resolved the issue.  I assume there was an incompatibility in one or both of the older versions of these libraries with python 3.7, but I had not previously realized that they were causing the issue.
History
Date User Action Args
2019-10-25 21:22:01kchensetstatus: open -> closed
resolution: not a bug
messages: + msg355387

stage: resolved
2019-10-24 06:15:27gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg355279
2019-10-24 01:50:19kchencreate