Title: Race condition during pickle.load()
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: papad, pitrou, ronaldoussoren, serhiy.storchaka, taleinat
Priority: normal Keywords:

Created on 2019-05-02 08:24 by papad, last changed 2019-08-05 08:26 by taleinat. This issue is now closed.

File name Uploaded Description Edit
reproduce.tar.gz papad, 2019-05-02 08:24
Messages (5)
msg341257 - (view) Author: papad (papad) Date: 2019-05-02 08:24
There seems to be a race condition when unpickling the same object from different threads (My guess is it's related to imports that are related to the unpickled object). 

I have used the following files for reproduction (File contents are in the attached archive, also pasted at the bottom of the issue report): - Containing the object we will pickle - the code to pickle the object - a piece of code to unpickle the object from different threads simultaneously and trigger the crash

I have used the files in the attached archive (containing the 3 files mentioned above)

to reproduce run the following commands:

Running will crash on about 50% of the runs with the following error:

Traceback (most recent call last):
  File "./", line 16, in pickle_load_thread
AttributeError: Can't get attribute 'PickleMe' on <module 'import_me' from '/data/'>

I have tested this on the following software stacks:

1. python:3.7.3 docker
    - linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 GNU/Linux
    - python: 3.7.3
    - distro: Debian GNU/Linux 9 (stretch)

2. my laptop
    - linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
    - python: 3.6.7
    - distro: Ubuntu 18.04.2 LTS

I'm uncertain if this is a bug related to pickle or python module importing.
Similar issues I've found: and

#! /usr/bin/python3

class PickleMe(object):

    def __init__(self):
        self.a = "a"

#! /usr/bin/python3

import pickle
from import_me import PickleMe

p = PickleMe()

with open('pickled', 'wb') as h:
    pickle.dump(p, h)

#! /usr/bin/python3

import threading
import pickle

import logging

threads = []

def pickle_load_thread():

    logging.error("Thread %d loading", threading.get_ident())

        with open('pickled', 'rb') as h:
        logging.exception("Exception in loading")

def start_pickle_load_thread():

    for x in range(2):
        load_thread = threading.Thread(target=pickle_load_thread)
        load_thread.daemon = True

    for x in threads:

if __name__ == '__main__':


    for t in threads:
msg341265 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2019-05-02 13:09
This could be a duplicate of Issue34572
msg341424 - (view) Author: papad (papad) Date: 2019-05-05 08:49
In what version should this be fixed? I see says the fix is in version 3.7, while I'm experiencing this on python 3.7.3, which version should I check?
msg341431 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-05-05 11:50
It should be fixed in 3.7.3.
msg349035 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-05 08:26
On Windows 10, this reproduces consistently with 3.6 and 3.7.0, but not with 3.7.4, 3.8.0b3 and current master. So this definitely seems to be fixed.
Date User Action Args
2019-08-05 08:26:43taleinatsetstatus: open -> closed

nosy: + taleinat
messages: + msg349035

resolution: fixed
stage: resolved
2019-05-05 11:50:52serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg341431
2019-05-05 08:49:31papadsetmessages: + msg341424
2019-05-02 13:09:00ronaldoussorensetnosy: + ronaldoussoren
messages: + msg341265
2019-05-02 11:17:28xtreaksetnosy: + pitrou
2019-05-02 08:24:59papadcreate