As I'm triangulating the problem down, I can now reproduce the problem without running django:

python -X faulthandler -c 'from asn1crypto._perf import _big_num_ctypes'

gives the output:

Fatal Python error: Aborted

Current thread 0x0000000117155dc0 (most recent call first):
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/__init__.py", line 364 in __init__
  File "/Users/laszlokarolyi/Work/project/venv/lib/python3.7/site-packages/asn1crypto/_perf/_big_num_ctypes.py", line 39 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 728 in exec_module
  File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1035 in _handle_fromlist
  File "<string>", line 1 in <module>
fish: 'python -X faulthandler -c 'from…' terminated by signal SIGABRT (Abort)

I took the opportunity to see what library loading fails, by printing self._name and mode before line 365 in ctypes/__init__.py:

It tries to load /usr/lib/libcrypto.dylib with mode 4, and that's where it crashes.