Author nadeem.vawda
Recipients alexandre.vassalotti, amaury.forgeotdarc, belopolsky, jorgsk, nadeem.vawda, nyevik, pitrou, santoso.wijaya
Date 2011-08-23.18:35:49
SpamBayes Score 1.21145e-08
Marked as misclassified No
Message-id <1314124550.61.0.389151959862.issue11564@psf.upfronthosting.co.za>
In-reply-to
Content
I was playing around with pickling large Unicode strings in an interactive
interpreter, and it seems that you have to have at least 4G chars (not bytes)
to trigger the OverflowError. Consider the following snippet of code:

    out = dumps(data)
    del data
    result = loads(out)
    assert isinstance(result, str)
    assert len(result) == _1G 

With data as (b"a" * _4G) the result is as expected:

    Traceback (most recent call last):
      File "pickle-bigmem-test.py", line 5, in <module>
        out = dumps(data)
    OverflowError: cannot serialize a string larger than 4GB

But with (b"a" * _2G), I get this:

    Traceback (most recent call last):
      File "pickle-bigmem-test.py", line 7, in <module>
        result = loads(out)
    _pickle.UnpicklingError: BINUNICODE pickle has negative byte count
History
Date User Action Args
2011-08-23 18:35:50nadeem.vawdasetrecipients: + nadeem.vawda, amaury.forgeotdarc, belopolsky, pitrou, alexandre.vassalotti, santoso.wijaya, nyevik, jorgsk
2011-08-23 18:35:50nadeem.vawdasetmessageid: <1314124550.61.0.389151959862.issue11564@psf.upfronthosting.co.za>
2011-08-23 18:35:50nadeem.vawdalinkissue11564 messages
2011-08-23 18:35:49nadeem.vawdacreate