Author tjb900
Recipients tjb900
Date 2018-09-03.15:53:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1535990001.55.0.56676864532.issue34572@psf.upfronthosting.co.za>
In-reply-to
Content
Retrieving and using a module directly from sys.modules (from C in this case) leads to a race condition where the module may be importing on another thread but has not yet been initialised.  For slow filesystems or large modules (e.g. numpy) this seems to lead to easily reproducible errors (the attached code fails 100% of the time on my work machine - CentOS 7).

I believe they have to be in sys.modules during this phase due to the possibility of circular references.

importlib handles this carefully with locking, but _pickle.c bypasses all that, leading to issues with threaded codes that use pickling, e.g. dask/distributed.
History
Date User Action Args
2018-09-03 15:53:21tjb900setrecipients: + tjb900
2018-09-03 15:53:21tjb900setmessageid: <1535990001.55.0.56676864532.issue34572@psf.upfronthosting.co.za>
2018-09-03 15:53:21tjb900linkissue34572 messages
2018-09-03 15:53:21tjb900create