Author Justin Ting
Recipients Justin Ting, serhiy.storchaka, tim.peters
Date 2016-10-22.16:53:23
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAPumCcMc4ZRWNpqHLDPGaw3PkJRM4EWXUb8gNF-x9icJ7HhK9w@mail.gmail.com>
In-reply-to <1477154913.3.0.618080276923.issue28506@psf.upfronthosting.co.za>
Content
Ah, should have picked that up, coding at 3:30am doesn't do wonders for
keeping a clear head.

Thanks Tim, I'll keep that in mind!

*Justin Ting*
*E* justingling@gmail.com |  *M* +61 424 751 665 | *L*
*https://au.linkedin.com/in/justinyting
<https://au.linkedin.com/in/justinyting>* | *G *https://github.com/jyting

On Sun, Oct 23, 2016 at 3:48 AM, Tim Peters <report@bugs.python.org> wrote:

>
> Tim Peters added the comment:
>
> This has nothing to do with the _values_ you're passing - it has to do
> with the length of the pickle string:
>
>     def _send_bytes(self, buf):
>         n = len(buf)
>         # For wire compatibility with 3.2 and lower
>         header = struct.pack("!i", n)  IT'S BLOWING UP HERE
>         if n > 16384:
>             ...
>             self._send(header)
>             self._send(buf)
>
> where the traceback shows it's called here:
>
>     self._send_bytes(ForkingPickler.dumps(obj))
>
> Of course the less data you're passing, the smaller the pickle, and that's
> why it doesn't blow up if you pass subsets of the data.
>
> I'd suggest rethinking how you're sharing data, as pushing two-gigabyte
> pickle strings around is bound to be the least efficient way possible even
> if it didn't blow up ;-)
>
> ----------
> nosy: +tim.peters
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28506>
> _______________________________________
>
History
Date User Action Args
2016-10-22 16:53:24Justin Tingsetrecipients: + Justin Ting, tim.peters, serhiy.storchaka
2016-10-22 16:53:24Justin Tinglinkissue28506 messages
2016-10-22 16:53:23Justin Tingcreate