Our hands are pretty much tied here. The pickle bytearray as unicode hack is likely the best we can do without pickling compatibility between Python 2 and 3. I can't think of a solution that could work here. For example.

1. Pickling bytearrays as a Python 2 str doesn't work because Python 2 strs are unpickled as unicode in Python 3.
2. Pickling bytearrays as an int lists makes the growth factor is much worst: 2x instead of the expected 1.5x.
3. Using a custom constructor breaks pickling compatibility with all the minor releases which doesn't implement the custom constructor.
4. Implementing special support in pickle for bytearrays requires a pickle protocol bump, which is disallowed for bugfixes releases.
5. Creating a special tag type to pickle Python 2 str as bytes in Python 3 has the same problem as #3.