Author sh
Recipients sh
Date 2009-11-23.15:17:32
SpamBayes Score 7.49401e-15
Marked as misclassified No
Message-id <1258989459.99.0.51081222067.issue7382@psf.upfronthosting.co.za>
In-reply-to
Content
In either python 3.0, bytes instances cannot be copied, and (even
trivial) bytes subclasses cannot be unpickled unless they explicitly
override __getnewargs__() or __reduce_ex__().

Copy problem:
>>> import copy; copy.copy(b'foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.1/copy.py", line 96, in copy
    return _reconstruct(x, rv, 0)
  File "/usr/lib/python3.1/copy.py", line 280, in _reconstruct
    y = callable(*args)
  File "/usr/lib/python3.1/copyreg.py", line 88, in __newobj__
    return cls.__new__(cls, *args)
TypeError: string argument without an encoding

Bytes subclass unpickle problem:
>>> class B(bytes):
...  pass
...
>>> import pickle; pickle.loads(pickle.dumps(B(b'foo')))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.1/pickle.py", line 1373, in loads
    encoding=encoding, errors=errors).load()
TypeError: string argument without an encoding


AFAICT, the problem is that bytes.__getnewargs__() returns a tuple with
a single argument - a string - and bytes.__new__() refuses to
reconstruct the instance when called with in that manner. That is,
"bytes.__new__(bytes, *b'foo'.__getnewargs__())" fails with a TypeError.

This does not cause a problem for pickling bytes instances (as opposed
to instances of a subclass of bytes), because both the Python and C
versions of pickle shipped with Python 3.[01] have built-in magic
(_Pickler.save_bytes() and save_bytes(), respectively) to deal with
bytes instances, and therefore never call their __getnewargs__().

The pickle case, in particular, is highly irritating; the error message
doesn't indicate which object is causing the problem, and until you
actually try to load the pickle, there's nothing to indicate that
there's anything problematic about pickling an instance of a subclass of
bytes.
History
Date User Action Args
2009-11-23 15:17:40shsetrecipients: + sh
2009-11-23 15:17:39shsetmessageid: <1258989459.99.0.51081222067.issue7382@psf.upfronthosting.co.za>
2009-11-23 15:17:34shlinkissue7382 messages
2009-11-23 15:17:33shcreate