Title: multiprocessing numpy.ndarray not transmitted properly
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: 2xB, davin
Priority: normal Keywords:

Created on 2019-07-20 18:11 by 2xB, last changed 2019-09-11 16:48 by davin. This issue is now closed.

Messages (3)
msg348218 - (view) Author: Benedikt Bieringer (2xB) Date: 2019-07-20 18:11
The following code demonstrates the issue:

import numpy as np
from multiprocessing import Pipe
p1, p2 = Pipe()
arr = np.zeros((3, 5, 6), dtype=np.uint8)
pm = p1.recv_bytes()

Only 3 bytes are transmitted for this array which obviously consists of more than 3 bytes.
msg351525 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2019-09-09 16:36
I believe you want to modify your sending of bytes to read:

The docs on send_bytes explains that it expects a bytes-like object.  NumPy arrays do not qualify by themselves but they can be readily converted.

To reconstruct that array from the 90 bytes received into the variable pm:
np.frombuffer(pm, dtype=np.int8).reshape((3, 5, 6))
msg351978 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2019-09-11 16:48
Marking as closed after providing an example of how to send NumPy arrays as bytes with the send_bytes() function.
Date User Action Args
2019-09-11 16:48:23davinsetstatus: closed
resolution: not a bug
messages: + msg351978

stage: resolved
2019-09-09 16:36:18davinsetstatus: open -> (no value)

messages: + msg351525
2019-07-21 05:31:22rhettingersetnosy: + davin
2019-07-20 18:11:382xBcreate