Le vendredi 06 janvier 2012 à 19:17 +0000, Charles-François Natali a
écrit :
> Charles-François Natali <neologix@free.fr> added the comment:
> 
> Antoine, could you test the last version (test_pickle and if possible
> with the OP testcase)?
> I can't test it myself (32-bit machine with 1 GB).

Well, first, there are compilation warnings on a 64-bit box:

gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -g -O0 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/usr/local/include -I/home/antoine/cpython/27/Include -I/home/antoine/cpython/27 -c /home/antoine/cpython/27/Modules/cPickle.c -o build/temp.linux-x86_64-2.7-pydebug/home/antoine/cpython/27/Modules/cPickle.o
/home/antoine/cpython/27/Modules/cPickle.c: In function ‘put2’:
/home/antoine/cpython/27/Modules/cPickle.c:810:9: attention : format ‘%d’ expects type ‘int’, but argument 4 has type ‘Py_ssize_t’
/home/antoine/cpython/27/Modules/cPickle.c: In function ‘load_mark’:
/home/antoine/cpython/27/Modules/cPickle.c:4610:21: attention : assignment from incompatible pointer type
gcc -pthread -shared build/temp.linux-x86_64-2.7-pydebug/home/antoine/cpython/27/Modules/cPickle.o -L/usr/local/lib -o build/lib.linux-x86_64-2.7-pydebug/cPickle.so

Second, I can't seem to get the test to work with 8GB RAM (approximately
6.5GB free according to "free"). The MemoryError is quite expectable for
test_pickle, though, since the code there doesn't try to conserve memory
at all:

test test_pickle failed -- Traceback (most recent call last):
  File "/home/antoine/cpython/27/Lib/test/test_support.py", line 983, in wrapper
    return f(self, maxsize)
  File "/home/antoine/cpython/27/Lib/test/pickletester.py", line 1298, in test_huge_str_32b
    pickled = self.dumps(data, proto)
  File "/home/antoine/cpython/27/Lib/test/test_pickle.py", line 74, in dumps
    return pickle.dumps(arg, proto)
  File "/home/antoine/cpython/27/Lib/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/home/antoine/cpython/27/Lib/pickle.py", line 224, in dump
    self.save(obj)
  File "/home/antoine/cpython/27/Lib/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/antoine/cpython/27/Lib/pickle.py", line 488, in save_string
    self.write(STRING + repr(obj) + '\n')
MemoryError

I would therefore suggest to enable the test only for cPickle.
For test_cpickle the behaviour is different:
- for protocol 0, I get a MemoryError (which may be expected, if the
test truly needs more than 6.5GB, for example because of suboptimal
buffer management)
- for protocol 1 and 2, I get "SystemError: error return without
exception set"