Author Justin Ting
Recipients Justin Ting
Date 2016-10-22.16:30:52
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1477153852.62.0.286648862234.issue28506@psf.upfronthosting.co.za>
In-reply-to
Content
Multiprocessing is throwing this error when dealing with large amounts of data (all floating points an integers), but none of which exceeds the number boundaries in the error that it throws:

  File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 268, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 385, in _handle_tasks
    put(task)
  File "/root/anaconda3/lib/python3.5/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/root/anaconda3/lib/python3.5/multiprocessing/connection.py", line 393, in _send_bytes
    header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
> /root/anaconda3/lib/python3.5/multiprocessing/connection.py(393)_send_bytes()
-> header = struct.pack("!i", n)

It works fine on any number of subsets of this data, but not when put together.
History
Date User Action Args
2016-10-22 16:30:52Justin Tingsetrecipients: + Justin Ting
2016-10-22 16:30:52Justin Tingsetmessageid: <1477153852.62.0.286648862234.issue28506@psf.upfronthosting.co.za>
2016-10-22 16:30:52Justin Tinglinkissue28506 messages
2016-10-22 16:30:52Justin Tingcreate