classification
Title: cPickle corner case - docs or bug?
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: alexandre.vassalotti Nosy List: alexandre.vassalotti, amaury.forgeotdarc, georg.brandl, gpk, llucax
Priority: normal Keywords:

Created on 2008-03-15 19:18 by gpk, last changed 2013-12-01 21:44 by alexandre.vassalotti. This issue is now closed.

Messages (5)
msg63560 - (view) Author: Greg Kochanski (gpk) Date: 2008-03-15 19:18
If you attempt to cPickle a class, cPickle checks
that it can get the identical class by importing it.

If that check fails, it reports:
Traceback (most recent call last):
...
"/usr/local/lib/python2.5/site-packages/newstem2-0.12.3-py2.5-linux-i686.egg/newstem2/stepperclient.py",
line 41, in send
    s = cPickle.dumps( args, cPickle.HIGHEST_PROTOCOL)
cPickle.PicklingError: Can't pickle <class 'test_simple2.aModel'>: it's
not the same object as test_simple2.aModel
$ 

Normally, this is probably a good thing.   However, if you do
an import using the "imp" module, via
 imp.load_module(name, fd, pn, desc), you get the "same" module
containing the "same" classes, but everything is duplicated at
different addresses.  In other words, you get distinct class
objects from what cPickle
will find.   Consequently, the when cPickle makes the
"is" comparison between what you gave it and what it can find,
it will fail and cause an error.

In this case, the error is wrong. I know that the aModel classes
come from the same file and are member-for-member the same.

This may well be a documentation
error:  it needs to mention this test and note that classes in modules
imported
via imp are not picklable.    Or, imp needs to note that its results
are not picklable.   Or both.

Or, maybe it's something that should be fixed, though I'm not
sure if there is a general solution that will always behave
well.
msg63629 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-03-17 10:09
The following "Works for me":

>>> import imp, cPickle
>>> mymod = imp.load_module('mymod', *imp.find_module('codecs'))
>>> cPickle.dumps(mymod.Codec(), cPickle.HIGHEST_PROTOCOL)
'\x80\x02(cmymod\nCodec\nq\x01o}q\x02b.'

Do you have a short test case to reproduce your problem?
Does your code tweak sys.modules?
msg68289 - (view) Author: Leandro Lucarella (llucax) Date: 2008-06-16 19:08
I'm having the same problem here.

Error is: cPickle.PicklingError: Can't pickle qos.Device: it's not the
same object as qos.Device

If I use pickle module, it works fine. If I use cPickle module but with
protocol=0, it works fine (protocol=1 or protocol=2 both fail).

I'm trying to make a simple example that reproduce the problem, but I
couldn't do it yet (small tests seems to work). So for now, I just can
only offer the complete source code of my project[1] (see branch
"python-bug-2295"). Not that the bug is exposed by the commit
b0ef5dd[2], and it's only exposed in the "qos" module (other modules
works fine).

To see the backtrace it should be enough to download the project (you
can download a tarball[3] or use git via git protocol[4] -preferred- or
http[5]) and execute ./pymind

[1]
http://git.llucax.com.ar/?p=software/pymin.git;a=shortlog;h=refs/heads/python-bug-2295
[2]
http://git.llucax.com.ar/?p=software/pymin.git;a=commitdiff;h=b0ef5ddfd5b04ecb09a18fb6f253c9f8a7db48a8
[3]
http://git.llucax.com.ar/?p=software/pymin.git;a=snapshot;h=f19fc5b4f5eeda18a24b1f5a2d042fd206c9ed0f;sf=tgz
[4] git clone git://git.llucax.com.ar/software/pymin.git
[5] git clone http://git.llucax.com.ar/git/software/pymin.git
msg68292 - (view) Author: Leandro Lucarella (llucax) Date: 2008-06-16 19:40
I've noted that the problem goes away if I move the pymin/services/*
directories (i.e., the packages in pymin/services) to another place (for
example, a services directory in the root of the project).

I still can't make a simple test to reproduce the problem.
msg204970 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2013-12-01 21:44
We can't fix this without a working test case. Feel free to re-open if you find one.
History
Date User Action Args
2013-12-01 21:44:07alexandre.vassalottisetstatus: open -> closed
versions: + Python 2.7, - Python 2.6, Python 3.1
messages: + msg204970

assignee: docs@python -> alexandre.vassalotti
components: - Documentation
resolution: works for me
2010-10-29 10:07:21adminsetassignee: georg.brandl -> docs@python
2009-05-16 19:38:23ajaksu2setpriority: normal
stage: test needed
versions: + Python 2.6, Python 3.1, - Python 2.5
2008-06-16 19:40:33llucaxsetmessages: + msg68292
2008-06-16 19:08:13llucaxsetnosy: + llucax
messages: + msg68289
2008-05-10 05:41:41alexandre.vassalottisetnosy: + alexandre.vassalotti
2008-03-17 10:09:44amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg63629
2008-03-15 19:18:44gpkcreate