Author tim.peters
Recipients Justin Ting, tim.peters
Date 2016-10-22.16:48:32
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1477154913.3.0.618080276923.issue28506@psf.upfronthosting.co.za>
In-reply-to
Content
This has nothing to do with the _values_ you're passing - it has to do with the length of the pickle string:

    def _send_bytes(self, buf):
        n = len(buf)
        # For wire compatibility with 3.2 and lower
        header = struct.pack("!i", n)  IT'S BLOWING UP HERE
        if n > 16384:
            ...
            self._send(header)
            self._send(buf)

where the traceback shows it's called here:

    self._send_bytes(ForkingPickler.dumps(obj))

Of course the less data you're passing, the smaller the pickle, and that's why it doesn't blow up if you pass subsets of the data.

I'd suggest rethinking how you're sharing data, as pushing two-gigabyte pickle strings around is bound to be the least efficient way possible even if it didn't blow up ;-)
History
Date User Action Args
2016-10-22 16:48:33tim.peterssetrecipients: + tim.peters, Justin Ting
2016-10-22 16:48:33tim.peterssetmessageid: <1477154913.3.0.618080276923.issue28506@psf.upfronthosting.co.za>
2016-10-22 16:48:33tim.peterslinkissue28506 messages
2016-10-22 16:48:32tim.peterscreate