This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author papad
Recipients papad
Date 2019-05-02.08:24:59
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1556785499.96.0.563265579216.issue36773@roundup.psfhosted.org>
In-reply-to
Content
There seems to be a race condition when unpickling the same object from different threads (My guess is it's related to imports that are related to the unpickled object). 

I have used the following files for reproduction (File contents are in the attached archive, also pasted at the bottom of the issue report):
import_me.py - Containing the object we will pickle
pickle_object.py - the code to pickle the object
trigger.py - a piece of code to unpickle the object from different threads simultaneously and trigger the crash

I have used the files in the attached archive (containing the 3 files mentioned above)

to reproduce run the following commands:

pickle_object.py
trigger.py

Running trigger.py will crash on about 50% of the runs with the following error:

Traceback (most recent call last):
  File "./trigger.py", line 16, in pickle_load_thread
    pickle.load(h)
AttributeError: Can't get attribute 'PickleMe' on <module 'import_me' from '/data/import_me.py'>


I have tested this on the following software stacks:

1. python:3.7.3 docker
    - linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 GNU/Linux
    - python: 3.7.3
    - distro: Debian GNU/Linux 9 (stretch)

2. my laptop
    - linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
    - python: 3.6.7
    - distro: Ubuntu 18.04.2 LTS


I'm uncertain if this is a bug related to pickle or python module importing.
Similar issues I've found: https://bugs.python.org/issue12680 and https://bugs.python.org/issue30891.


---------------------------------------------------------------------------------------
filename: import_me.py
#! /usr/bin/python3


class PickleMe(object):

    def __init__(self):
        self.a = "a"

---------------------------------------------------------------------------------------
filename: pickle_object.py
#! /usr/bin/python3

import pickle
from import_me import PickleMe

p = PickleMe()

with open('pickled', 'wb') as h:
    pickle.dump(p, h)

---------------------------------------------------------------------------------------
filename: trigger.py
#! /usr/bin/python3

import threading
import pickle

import logging

threads = []

def pickle_load_thread():

    logging.error("Thread %d loading", threading.get_ident())

    try:
        with open('pickled', 'rb') as h:
            pickle.load(h)
    except:
        logging.exception("Exception in loading")


def start_pickle_load_thread():

    for x in range(2):
        load_thread = threading.Thread(target=pickle_load_thread)
        load_thread.daemon = True
        threads.append(load_thread)

    for x in threads:
        x.start()


if __name__ == '__main__':

    start_pickle_load_thread()

    for t in threads:
        t.join()
History
Date User Action Args
2019-05-02 08:24:59papadsetrecipients: + papad
2019-05-02 08:24:59papadsetmessageid: <1556785499.96.0.563265579216.issue36773@roundup.psfhosted.org>
2019-05-02 08:24:59papadlinkissue36773 messages
2019-05-02 08:24:59papadcreate