Message279203
Ah, should have picked that up, coding at 3:30am doesn't do wonders for
keeping a clear head.
Thanks Tim, I'll keep that in mind!
*Justin Ting*
*E* justingling@gmail.com | *M* +61 424 751 665 | *L*
*https://au.linkedin.com/in/justinyting
<https://au.linkedin.com/in/justinyting>* | *G *https://github.com/jyting
On Sun, Oct 23, 2016 at 3:48 AM, Tim Peters <report@bugs.python.org> wrote:
>
> Tim Peters added the comment:
>
> This has nothing to do with the _values_ you're passing - it has to do
> with the length of the pickle string:
>
> def _send_bytes(self, buf):
> n = len(buf)
> # For wire compatibility with 3.2 and lower
> header = struct.pack("!i", n) IT'S BLOWING UP HERE
> if n > 16384:
> ...
> self._send(header)
> self._send(buf)
>
> where the traceback shows it's called here:
>
> self._send_bytes(ForkingPickler.dumps(obj))
>
> Of course the less data you're passing, the smaller the pickle, and that's
> why it doesn't blow up if you pass subsets of the data.
>
> I'd suggest rethinking how you're sharing data, as pushing two-gigabyte
> pickle strings around is bound to be the least efficient way possible even
> if it didn't blow up ;-)
>
> ----------
> nosy: +tim.peters
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28506>
> _______________________________________
> |
|
Date |
User |
Action |
Args |
2016-10-22 16:53:24 | Justin Ting | set | recipients:
+ Justin Ting, tim.peters, serhiy.storchaka |
2016-10-22 16:53:24 | Justin Ting | link | issue28506 messages |
2016-10-22 16:53:23 | Justin Ting | create | |
|