This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Unpickling derived class of list does not call __init__()
Type: Stage: resolved
Components: Versions: Python 3.8
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: andymaier, gregory.p.smith, pitrou, serhiy.storchaka
Priority: normal Keywords:

Created on 2020-08-26 06:50 by andymaier, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg375909 - (view) Author: Andy Maier (andymaier) * Date: 2020-08-26 06:50
Unpickling an object of a user class that derives from list seems to miss calling the user class's __init__() method:

Consider this script, which defines a derived class of the built-in list for the purpose of creating a case insensitive list. The real example has all methods of list implemented, but this subset example also shows the issue:

import pickle

def _lc_list(lst):
    result = list()
    for value in lst:
    return result

class NocaseList(list):

    #def __new__(cls, iterable=()):
    #    self = super(NocaseList, cls).__new__(cls, iterable)
    #    print("Debug: __new__()  for self={}".format(id(self)))
    #    return self

    def __init__(self, iterable=()):
        print("Debug: __init__() for self={}".format(id(self)))
        super(NocaseList, self).__init__(iterable)
        self.lc_list = _lc_list(self)

    def append(self, value):
        super(NocaseList, self).append(value)

    def extend(self, iterable):
        super(NocaseList, self).extend(iterable)
        for value in iterable:

ncl = NocaseList(['A', 'b'])
pkl = pickle.dumps(ncl)
nclx = pickle.loads(pkl)

$ python ./ 
Debug: __init__() for self=4498704880
Traceback (most recent call last):
  File "./", line 32, in <module>
    nclx = pickle.loads(pkl)
  File "./", line 28, in extend
AttributeError: 'NocaseList' object has no attribute 'lc_list'

Uncommenting the __new__() method in the script shows that __new__() is called but not __init__():

$ python ./ 
Debug: __new__()  for self=4359683024
Debug: __init__() for self=4359683024
Debug: __new__()  for self=4360134304
Traceback (most recent call last):
  File "./", line 32, in <module>
    nclx = pickle.loads(pkl)
  File "./", line 28, in extend
AttributeError: 'NocaseList' object has no attribute 'lc_list'
msg375929 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-08-26 11:02

    When a class instance is unpickled, its __init__() method is usually not invoked.

If you want to change this behavior you have to implement the __reduce__ or __reduce_ex__ methods or register the object type in the global or per-pickler dispatch table. For example:

class NocaseList(list):

    def __reduce__(self):
        return self.__class__, (), None, iter(self)
msg376525 - (view) Author: Andy Maier (andymaier) * Date: 2020-09-07 19:20
Thanks for the clarification.

Just for the record:

I have implemented __setstate__() such that it completely restores the state from just the inherited list state. That makes it independent of whether __init__() or __new__() is called:

    def __getstate__(self):
        state = self.__dict__.copy()
        del state['_lc_list']
        return state

    def __setstate__(self, state):
        self._lc_list = _lc_list(self)

This issue can be closed.
msg376527 - (view) Author: Andy Maier (andymaier) * Date: 2020-09-07 19:21
And just to be complete: That did not require implementing __reduce__().
msg392002 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-04-27 00:05
this looks resolved?
Date User Action Args
2022-04-11 14:59:35adminsetgithub: 85805
2021-04-27 00:05:24gregory.p.smithsetstatus: open -> closed

nosy: + gregory.p.smith
messages: + msg392002

resolution: not a bug
stage: resolved
2020-09-07 19:21:49andymaiersetmessages: + msg376527
2020-09-07 19:20:50andymaiersetmessages: + msg376525
2020-08-26 11:02:40serhiy.storchakasetmessages: + msg375929
2020-08-26 08:24:13xtreaksetnosy: + pitrou, serhiy.storchaka
2020-08-26 06:50:59andymaiercreate