It's interesting that you would not count a critical segfault in Pickle as a threat, because there are numerous libraries that are Unpickling untrusted user data (even-though some of them are using RestrictedUnpickler to protect themselves but a segfault would bypass that). For example, Ray Project with five thousands commits (https://github.com/ray-project/ray/blob/master/rllib/utils/policy_server.py#L31). 

Long story short, you advise us to not put time on checking the security of the Pickle module too, am I right?

Thanks,
Iman