On 5/10/2013 11:46 PM, Stefan Mihaila wrote:
> Changes by Stefan Mihaila <mstefanro@gmail.com>:
>
>
> ----------
> nosy: +mstefanro
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue17810>
> _______________________________________
>
Hello. I've worked on implementing PEP3154 as part of GSoC2012.
My work is available in a repo at [1].
The blog I've used to report my work is at [2] and contains some useful 
information.

Here is a list of features that were implemented as part of GSoC:

* Pickling of very large bytes and strings
* Better pickling of small string and bytes (+ tests)
* Native pickling of sets and frozensets (+ tests)
* Self-referential sets and frozensets (+ tests)
* Implicit memoization (BINPUT is implicit for certain opcodes)
   - The argument against this was that pickletools.optimize would
     not be able to prevent memoization of objects that are not
     referred later. For such situations, a special flag at beginning
     could be added, which indicates whether implicit BINPUT is enabled.
     This flag could be added as one of the higher-order bits of the 
protocol
     version. For instance:
         PROTO \x04 + BINUNICODE ".."
         and
         PROTO \x84 + BINUNICODE ".." + BINPUT 1
     would be equivalent. Then pickletools.optimize could choose whether
     it wants implicit BINPUT or not. Sure, this would complicate 
matters and it's
     not for me to decide whether it's worth it.
     In my midterm report at [3] there are some examples of what a 
pickled string
     looks in v4 without implicit memoization, and some size comparisons
     to v3.
* Pickling of nested globals, methods etc. (+ tests)
* Pickling calls to __new__ with keyword args (+ tests)
* A BAIL_OUT opcode was always outputted when pickling failed, so that
   the Pickler and Unpickler can be both run at once on different ends
   of a stream. The Pickler could guarantee to always send a
   correct pickle on the stream. The Unpickler would never end up hanging
   when Pickling failed mid-work.
   -  At the time, Alexandre suggested this would probably not be a great
      idea because it should be the responsibility of the protocol used
      to assure some consistency. However, this does not appear to be
      a trivial task to achieve. The size of the pickle is not known in
      advance, and waiting for the Pickler to complete before sending
      the data via stream is not as efficient, because the Unpickler
      would not be able to run at the same time.
      write and read methods of the stream would have to be wrapped and
      some escape sequence used. This would
      increase the size of the pickled string for some sort of worst-case
      of the escape sequence, probably. My thought was that it would be
      beneficial for the average user to have the guarantee that the Pickler
      always outputs a correct pickle to a stream, even if it raises an 
exception.
* Other minor changes that I can't really remember.

Although I'm sure Alexandre had his good reasons to start the work from
scratch, it would be a shame to waste all this work. The features mentioned
above are working and although the implementation may not be ideal (I don't
have the cpython experience of a regular dev), I'm sure useful bits can be
extracted from it.
Alexandre suggested that I extract bits and post patches, so I have 
attached,
for now, support for pickling methods and nested globals (+tests).
I'm willing to do so for some or the rest of the features, should this 
be requested
and should I have the necessary time to do so.

[1] https://bitbucket.org/mstefanro/pickle4/
[2] https://pypickle4.wordpress.com/
[3] https://gist.github.com/mstefanro/3086647