Message401219
As we've discussed in https://bugs.python.org/msg401146 we need to improve how `SharedMemory` is documented and tested. Right now docs do not cover `pickle` at all (https://github.com/python/cpython/blob/main/Doc/library/multiprocessing.shared_memory.rst). And this is an important aspect in multiprocessing.
`SharedMemory` and `pickle` in `test_shared_memory_basics` (https://github.com/python/cpython/blob/a5c6bcf24479934fe9c5b859dd1cf72685a0003a/Lib/test/_test_multiprocessing.py#L3789-L3794):
```
sms.buf[0:6] = b'pickle'
pickled_sms = pickle.dumps(sms)
sms2 = pickle.loads(pickled_sms)
self.assertEqual(sms.name, sms2.name)
self.assertEqual(bytes(sms.buf[0:6]), bytes(sms2.buf[0:6]), b'pickle')
```
`ShareableList` has better coverage in this regard (https://github.com/python/cpython/blob/a5c6bcf24479934fe9c5b859dd1cf72685a0003a/Lib/test/_test_multiprocessing.py#L4121-L4144):
```
def test_shared_memory_ShareableList_pickling(self):
sl = shared_memory.ShareableList(range(10))
self.addCleanup(sl.shm.unlink)
serialized_sl = pickle.dumps(sl)
deserialized_sl = pickle.loads(serialized_sl)
self.assertTrue(
isinstance(deserialized_sl, shared_memory.ShareableList)
)
self.assertTrue(deserialized_sl[-1], 9)
self.assertFalse(sl is deserialized_sl)
deserialized_sl[4] = "changed"
self.assertEqual(sl[4], "changed")
# Verify data is not being put into the pickled representation.
name = 'a' * len(sl.shm.name)
larger_sl = shared_memory.ShareableList(range(400))
self.addCleanup(larger_sl.shm.unlink)
serialized_larger_sl = pickle.dumps(larger_sl)
self.assertTrue(len(serialized_sl) == len(serialized_larger_sl))
larger_sl.shm.close()
deserialized_sl.shm.close()
sl.shm.close()
```
So, my plan is:
1. Improve testing of `SharedMemory` after pickling/unpickling. I will create a separate test with all the `pickle`-related stuff there
2. Improve docs: user must understand what will have when `SharedMemory` / `SharableList` is pickled and unpickled. For example, the fact that it will still be shared can be a surprise to many.
I am going to send a PR with both thing somewhere this week.
I will glad to head any feedback before / after :) |
|
Date |
User |
Action |
Args |
2021-09-07 10:17:52 | sobolevn | set | recipients:
+ sobolevn, docs@python |
2021-09-07 10:17:52 | sobolevn | set | messageid: <1631009872.94.0.369430162871.issue45125@roundup.psfhosted.org> |
2021-09-07 10:17:52 | sobolevn | link | issue45125 messages |
2021-09-07 10:17:52 | sobolevn | create | |
|