classification
Title: Infinite Recursion during Unpickling a codecs Object
Type: feature request Stage:
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ThomasH, alexandre.vassalotti, lemburg (3)
Priority: Keywords

Created on 2009-07-01 15:46 by ThomasH, last changed 2009-07-08 08:12 by ThomasH.

Messages (3)
msg89985 - (view) Author: (ThomasH) Date: 2009-07-01 15:46
I recently ran into an infinite recursion trying to unpickle a
codecs.StreamWriter object (I presume the issue would be the same for a
StreamReader).

Here is the end of the stack trace:

  File "/sw/lib/python2.5/codecs.py", line 330, in __getattr__    
    return getattr(self.stream, name)
  File "/sw/lib/python2.5/codecs.py", line 330, in __getattr__  
    return getattr(self.stream, name) 
  File "/sw/lib/python2.5/codecs.py", line 330, in __getattr__ 
    return getattr(self.stream, name) 
 RuntimeError: maximum recursion depth exceeded 

The issue is the same under Python2.6 but the error output has changed
(see http://bugs.python.org/issue5508).

The problem is that the codecs module tries to delegate member lookup to
the underlying stream. But after unpickling, "self.stream" is not
defined, so the reference to self.stream in __getattr__ itself leads to
an invocation of __getattr__ - hence the recursion loop.

Using tools from the Pickle protocol, like __getstate__/__setstate__,
could help degrade codecs objects gracefully during pickle/unpickle
cycles. E.g. it might be enough to provide a dummy self.stream through
__setstate__.
msg90180 - (view) Author: Marc-Andre Lemburg (lemburg) Date: 2009-07-06 14:26
I don't understand the use case for this. 

If the StreamWriter/Reader cannot pickle the underlying stream (which is
probably always the case), why should the object itself be pickleable ?

What we could do is implement .__get/setstate__() and have it raise an
exception to inform the user of the problem.
msg90258 - (view) Author: (ThomasH) Date: 2009-07-08 08:12
Sounds good to me. The main intention of this bug is not to make codecs
objects pickleable by all means (I don't know if this would be sensible
and/or possible), but at least to have them degrade gracefully during
pickling, particularly without infinite recursion. I've changed the bug
title to reflect this.
History
Date User Action Args
2009-07-08 08:12:19ThomasHsetmessages: + msg90258
title: Add Pickle Support to the codecs Module -> Infinite Recursion during Unpickling a codecs Object
2009-07-06 14:26:40lemburgsetnosy: + lemburg
messages: + msg90180
2009-07-03 00:03:29alexandre.vassalottisetnosy: + alexandre.vassalotti
2009-07-01 16:02:16benjamin.petersonsettype: crash -> feature request
versions: + Python 2.7, Python 3.2, - Python 2.6, Python 2.5
2009-07-01 15:46:26ThomasHcreate