Hi,

sometimes (but not reliably reproducibly, one has to run it a few times) I get a segmentation fault when running the following networkx based Python code on large input graphs: https://gitlab.mister-muffin.de/debian-bootstrap/botch/blob/master/tools/graph-difference.py

I'm running Debian unstable with the python3.4 package of version 3.4.3-7 on architecture amd64.

The core dump is 1GB large so I'm just attaching the traceback from gdb.

The string "hscolour:amd64 (= 1.20.3-2)" that you see in the traceback is one of the vertex attributes in the input graph.

What else do you need to debug the problem?

#0  asciilib_split_char.lto_priv () at ../Objects/stringlib/split.h:126
#1  0x000000000058e65a in asciilib_split (maxcount=<optimized out>, sep_len=1, sep=0x7f1b3088dfb0 ",", str_len=27, 
    str=0x7f1b230abfb0 "hscolour:amd64 (= 1.20.3-2)", str_obj='hscolour:amd64 (= 1.20.3-2)') at ../Objects/stringlib/split.h:158
#2  split (maxcount=<optimized out>, substring=',', self='hscolour:amd64 (= 1.20.3-2)') at ../Objects/unicodeobject.c:10099
#3  unicode_split.lto_priv () at ../Objects/unicodeobject.c:12639
#4  0x000000000050d8fe in call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8ed80) at ../Python/ceval.c:4237
#5  PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#6  0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#7  0x000000000051163d in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7ffdf1a8ef60, 
    func=<optimized out>) at ../Python/ceval.c:4344
#8  call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8ef60) at ../Python/ceval.c:4262
#9  PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#10 0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#11 0x000000000051163d in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7ffdf1a8f140, 
    func=<optimized out>) at ../Python/ceval.c:4344
#12 call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8f140) at ../Python/ceval.c:4262
#13 PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#14 0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#15 0x000000000051163d in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7ffdf1a8f320, 
    func=<optimized out>) at ../Python/ceval.c:4344
#16 call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8f320) at ../Python/ceval.c:4262
#17 PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#18 0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#19 0x00000000005e16a5 in PyEval_EvalCode (
    locals={'__package__': None, '__doc__': None, '__spec__': None, 'sys': <module at remote 0x7f1b309120e8>, 'graph_difference': <function at remote 0x7f1b308ddc80>, '__file__': './tools/graph-difference.py', '__builtins__': <module at remote 0x7f1b31cd2c28>, 'parser': <ArgumentParser(_action_groups=[<_ArgumentGroup(_action_groups=[], _has_negative_number_optionals=[], prefix_chars='-', description=None, _defaults={}, _negative_number_matcher=<_sre.SRE_Pattern at remote 0x7f1b2e002d98>, _registries={'action': {'append': <type at remote 0x2d57a58>, 'store_true': <type at remote 0x2d548a8>, 'store_false': <type at remote 0x2d54da8>, 'help': <type at remote 0x2d58688>, 'count': <type at remote 0x2d58278>, 'append_const': <type at remote 0x2d57e68>, 'store': <type at remote 0x2ded3b8>, None: <type at remote 0x2ded3b8>, 'store_const': <type at remote 0x2d54498>, 'version': <type at remote 0x2d58bb8>, 'parsers': <type at remote 0x2d5df78>}, 'type': {None: <function at remote 0x7f1b2d1f86a8>}}, _group_actions=[<_StoreAction(d...(truncated), 
    globals={'__package__': None, '__doc__': None, '__spec__': None, 'sys': <module at remote 0x7f1b309120e8>, 'graph_difference': <function at remote 0x7f1b308ddc80>, '__file__': './tools/graph-difference.py', '__builtins__': <module at remote 0x7f1b31cd2c28>, 'parser': <ArgumentParser(_action_groups=[<_ArgumentGroup(_action_groups=[], _has_negative_number_optionals=[], prefix_chars='-', description=None, _defaults={}, _negative_number_matcher=<_sre.SRE_Pattern at remote 0x7f1b2e002d98>, _registries={'action': {'append': <type at remote 0x2d57a58>, 'store_true': <type at remote 0x2d548a8>, 'store_false': <type at remote 0x2d54da8>, 'help': <type at remote 0x2d58688>, 'count': <type at remote 0x2d58278>, 'append_const': <type at remote 0x2d57e68>, 'store': <type at remote 0x2ded3b8>, None: <type at remote 0x2ded3b8>, 'store_const': <type at remote 0x2d54498>, 'version': <type at remote 0x2d58bb8>, 'parsers': <type at remote 0x2d5df78>}, 'type': {None: <function at remote 0x7f1b2d1f86a8>}}, _group_actions=[<_StoreAction(d...(truncated), co=<code at remote 0x7f1b307f09c0>) at ../Python/ceval.c:775
#20 run_mod () at ../Python/pythonrun.c:2180
#21 0x00000000005e176a in PyRun_FileExFlags () at ../Python/pythonrun.c:2133
#22 0x00000000005e237a in PyRun_SimpleFileExFlags () at ../Python/pythonrun.c:1606
#23 0x00000000005fdb60 in run_file (p_cf=<optimized out>, filename=<optimized out>, fp=<optimized out>) at ../Modules/main.c:319
#24 Py_Main () at ../Modules/main.c:751
#25 0x00000000004c234f in main () at ../Modules/python.c:69
#26 0x00007f1b30972b45 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#27 0x00000000005ba765 in _start ()

It looks more like a bug in networkx, than a bug in Python itself. networkx has probably issues with reference counter, concurrency (threads), or things like that.

I'm unable to reproduce the crash on Python 3.4 (system binary from Fedora 22) or Python 3.6 (compiled manually).


haypo@smithers$ ./python
Python 3.6.0a0 (default:bb9fc884a838, Jul  6 2015, 15:27:15) 
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux
>>> "hscolour:amd64 (= 1.20.3-2)".split()
['hscolour:amd64', '(=', '1.20.3-2)']
>>> "hscolour:amd64 (= 1.20.3-2)".split(",")
['hscolour:amd64 (= 1.20.3-2)']
>>> len("hscolour:amd64 (= 1.20.3-2)")
27


haypo@smithers$ python3
Python 3.4.2 (default, Jan 12 2015, 12:13:20) 
[GCC 4.9.2 20150107 (Red Hat 4.9.2-5)] on linux
>>> "hscolour:amd64 (= 1.20.3-2)".split()
['hscolour:amd64', '(=', '1.20.3-2)']
>>> "hscolour:amd64 (= 1.20.3-2)".split(",")
['hscolour:amd64 (= 1.20.3-2)']
>>> len("hscolour:amd64 (= 1.20.3-2)")
27

Oh, networkx looks to be written in pure Python. You should search for a module implemented in C.

I do not see any module implemented in C in the imports. Is there a way to find out from where the segmentation fault came?

You have to search for memory corruptions. You can try to run your application with a Python compiled a debug mode. If it doesn't work, you may try Valgrind which require a Python compiled with --with-valgrind and to use the suppression file. See Misc/README.valgrind.

Is there anything we can do about this or should it be closed?

No activity for 6 years, the reporter didn't provide any way to reproduce the crash, I close the issue.

Note: Python 3.10 now dumps the list of third party C extensions on a fatal error, or when faulthandler handlers a crash ;-) Just enabled the Python Development Mode:
https://docs.python.org/dev/library/devmode.html