classification
Title: create multiprocessing.SharedMemory by pointing to existing memoryview
Type: performance Stage:
Components: C API Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Dariusz Trawinski
Priority: normal Keywords:

Created on 2020-02-26 23:46 by Dariusz Trawinski, last changed 2020-02-26 23:46 by Dariusz Trawinski.

Messages (1)
msg362754 - (view) Author: Dariusz Trawinski (Dariusz Trawinski) Date: 2020-02-26 23:46
Currently, in order to share numpy array between processes via multiprocessing.SharedMemory object, it is required to copy the memory content with:
input = np.ones((1,10,10,10))
shm = shared_memory.SharedMemory(create=True, size=input.nbytes)
write_array = np.ndarray(input.shape, dtype=input.dtype,buffer=shm.buf)
write_array1[:] = input[:]
In result the original numpy array is duplicated in RAM. It also adds extra cpu cycles to copy the content.

I would like to recommend adding an option to create shared memory object by pointing it to existing memoryview object, beside current method of using shared memory name. 
Is that doable?
History
Date User Action Args
2020-02-26 23:46:31Dariusz Trawinskicreate