classification
Title: Crash in set_traverse Within the Garbage Collector's collect_generations()
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: connorwfitzgerald, pitrou
Priority: normal Keywords:

Created on 2017-12-04 03:37 by connorwfitzgerald, last changed 2018-04-26 02:21 by connorwfitzgerald. This issue is now closed.

Messages (3)
msg307539 - (view) Author: Connor W Fitzgerald (connorwfitzgerald) Date: 2017-12-04 03:37
This is my first time submitting a bug report, so forgive me if I miss on important information. I am writing code that makes heavy use of sets (8GB+ of them). It segfaults semi-randomly during the processing of them. I've used versions 3.5.3, 3.6.3, and 3.7.0rc2. I debugged the process and came across the error in object/setobject.c:742. I don't really know what information to provide here but this is the best I can do.

Error:
Unhandled exception thrown: read access violation.
entry->**key** was 0x8000.

Stack Trace:
python37.dll!set_traverse(PySetObject * so, int(*)(_object *, void *) visit, void * arg) Line 742
python37.dll!subtract_refs(_gc_head * containers) Line 295
python37.dll!collect(int generation, __int64 * n_collected, __int64 * n_uncollectable, int nofail) Line 862
python37.dll!collect_with_callback(int generation) Line 1027
python37.dll!collect_generations() Line 1049
python37.dll!PyType_GenericAlloc(_typeobject * type, __int64 nitems) Line 969
[Inline Frame] python37.dll!make_new_set(_typeobject *) Line 1055
python37.dll!set_new(_typeobject * type, _object * args, _object * kwds) Line 1127
python37.dll!type_call(_typeobject * type, _object * args, _object * kwds) Line 928
python37.dll!_PyObject_FastCallKeywords(_object * callable, _object * * stack, __int64 nargs, _object * kwnames) Line 199
python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4705
python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3182
[Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 549
python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * * args, __int64 argcount, _object * * kwnames, _object * * kwargs, __int64 kwcount, int kwstep, _object * * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4011
python37.dll!_PyFunction_FastCallDict(_object * func, _object * * args, __int64 nargs, _object * kwargs) Line 376
[Inline Frame] python37.dll!PyEval_CallObjectWithKeywords(_object *) Line 819
python37.dll!defdict_missing(defdictobject * dd, _object * key) Line 1993
python37.dll!_PyMethodDef_RawFastCallDict(PyMethodDef * method, _object * self, _object * * args, __int64 nargs, _object * kwargs) Line 496
[Inline Frame] python37.dll!_PyCFunction_FastCallDict(_object *) Line 580
python37.dll!_PyObject_FastCallDict(_object * callable, _object * * args, __int64 nargs, _object * kwargs) Line 101
[Inline Frame] python37.dll!object_vacall(_object * callable, char *) Line 1194
python37.dll!PyObject_CallFunctionObjArgs(_object * callable, ...) Line 1259
python37.dll!dict_subscript(PyDictObject * mp, _object * key) Line 1984
python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 1316
[Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 549
python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * * args, __int64 argcount, _object * * kwnames, _object * * kwargs, __int64 kwcount, int kwstep, _object * * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4011
[Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, _object * *) Line 433
python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * kwnames) Line 4703
python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3182
[Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 549
python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * * args, __int64 argcount, _object * * kwnames, _object * * kwargs, __int64 kwcount, int kwstep, _object * * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4011
python37.dll!PyEval_EvalCodeEx(_object * _co, _object * globals, _object * locals, _object * * args, int argcount, _object * * kws, int kwcount, _object * * defs, int defcount, _object * kwdefs, _object * closure) Line 4045
python37.dll!PyEval_EvalCode(_object * co, _object * globals, _object * locals) Line 532
python37.dll!run_mod(_mod * mod, _object * filename, _object * globals, _object * locals, PyCompilerFlags * flags, _arena * arena) Line 987
python37.dll!PyRun_FileExFlags(_iobuf * fp, const char * filename_str, int start, _object * globals, _object * locals, int closeit, PyCompilerFlags * flags) Line 939
python37.dll!PyRun_SimpleFileExFlags(_iobuf * fp, const char * filename, int closeit, PyCompilerFlags * flags) Line 402
python37.dll!PyRun_AnyFileExFlags(_iobuf * fp, const char * filename, int closeit, PyCompilerFlags * flags) Line 84
python37.dll!run_file(_iobuf * fp, const wchar_t * filename, PyCompilerFlags * p_cf) Line 340
python37.dll!Py_Main(int argc, wchar_t * * argv) Line 894
[Inline Frame] python.exe!invoke_main() Line 79
python.exe!__scrt_common_main_seh() Line 253
kernel32.dll!0000000077a959cd()
ntdll.dll!0000000077bca561()

I'm not sure what I can do to reproduce this because I don't even know what line of code my program was running before it segfaulted. I'm using sqlite3, json, and pickle as my main extensions. Nothing outside of what is provided in the standard library is used. If you need any more information, I'll gladly provide it.
msg307871 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-12-08 21:46
> I'm not sure what I can do to reproduce this because I don't even know what line of code my program was running before it segfaulted. 

To do that you could enable the faulthandler module: https://docs.python.org/3/library/faulthandler.html

Also, it would be nice if you had a simple reproducer to share with us, though it may be difficult given the description you gave of your workload.
msg315770 - (view) Author: Connor W Fitzgerald (connorwfitzgerald) Date: 2018-04-26 02:21
I'm going to close this. It's been a really long time and this was for an assignment and I rewrote it before we put it into a github repo so I don't even have the buggy code still. Additionally I think it may have been the fault of some faulty memory I had in my computer so I'm betting that it wasn't pythons fault. Cheers.
History
Date User Action Args
2018-04-26 02:21:16connorwfitzgeraldsetstatus: open -> closed
resolution: not a bug
stage: resolved
2018-04-26 02:21:03connorwfitzgeraldsetmessages: + msg315770
2017-12-08 21:46:31pitrousetnosy: + pitrou
messages: + msg307871
2017-12-04 03:37:11connorwfitzgeraldcreate