Message362291
So I *think* I've pieced together what caused the user crash that originated in the flair library. It turns out that pickle.load, via torch.load, is getting passed an mmap.mmap.
https://github.com/flairNLP/flair/blob/1d44abf35f1122d1e146c9a219b2cb5f98b41b40/flair/file_utils.py#L25-L36
https://github.com/flairNLP/flair/blob/1d44abf35f1122d1e146c9a219b2cb5f98b41b40/flair/nn.py#L85-L86
Since mmap doesn't implement readinto, pickle.load objects as of Python 3.8. This is new behavior in Python3.8, it used to be possible to load a memory-mapped pickle file.
Short repro script:
import pickle
import mmap
data = "some data"
with open('my_data.pkl', 'wb') as f:
pickle.dump(data, f)
with open("my_data.pkl", "r+b") as f_in:
mm = mmap.mmap(f_in.fileno(), 0)
print(pickle.load(mm))
On Python3.8, this script prints an error, on Python3.7 it prints "some data". |
|
Date |
User |
Action |
Args |
2020-02-19 21:44:28 | Nathan.Goldbaum | set | recipients:
+ Nathan.Goldbaum, pitrou |
2020-02-19 21:44:28 | Nathan.Goldbaum | set | messageid: <1582148668.53.0.428067131933.issue39681@roundup.psfhosted.org> |
2020-02-19 21:44:28 | Nathan.Goldbaum | link | issue39681 messages |
2020-02-19 21:44:28 | Nathan.Goldbaum | create | |
|