This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: struct objects can no longer be pickled
Type: crash Stage: resolved
Components: Windows Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: mesheb82, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2017-02-23 03:55 by mesheb82, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg288401 - (view) Author: mesheb82 (mesheb82) Date: 2017-02-23 03:55
When I run the following code on Windows/Linux for < Python 3.6, I have no problems.  When I run in Python 3.6.0, I get the subsequent traceback.

I checked the release notes and only saw the following struct module note related to half-floats: Issue #11734

from struct import Struct
import copy
this_fails = Struct('<i')
copy.deepcopy(this_fails)


Traceback (most recent call last):
  File "test_script.py", line 12, in <module>
    copy.deepcopy(this_fails)
  File "F:\Anaconda\envs\py36\lib\copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle Struct objects


To be clear, I'm copying struct objects as part of a larger class.  As I'm running many function calls where I create the same struct objects, saving the common struct objects allows me to get a factor of 2x speedup on my code.  Deleting the saved struct objects at the end of file reading (and before the deepcopy takes place) fixes the problem.
msg288406 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-23 05:22
> When I run the following code on Windows/Linux for < Python 3.6, I have no problems.

You have a problem. Did you try to use the copied object? It is a broken Struct object.

>>> copied = copy.deepcopy(this_fails)
>>> copied.format
>>> copied.size
-1
>>> copied.pack(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: pack expected -1 items for packing (got 1)
>>> copied.unpack(b'abcd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: unpack requires a bytes object of length -1

Copying Struct object never worked. Now it raises an error rather than silently creating a broken object.
msg288407 - (view) Author: mesheb82 (mesheb82) Date: 2017-02-23 05:27
Thank you.

On Wed, Feb 22, 2017 at 9:22 PM, Serhiy Storchaka <report@bugs.python.org>
wrote:

>
> Serhiy Storchaka added the comment:
>
> > When I run the following code on Windows/Linux for < Python 3.6, I have
> no problems.
>
> You have a problem. Did you try to use the copied object? It is a broken
> Struct object.
>
> >>> copied = copy.deepcopy(this_fails)
> >>> copied.format
> >>> copied.size
> -1
> >>> copied.pack(42)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> struct.error: pack expected -1 items for packing (got 1)
> >>> copied.unpack(b'abcd')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> struct.error: unpack requires a bytes object of length -1
>
> Copying Struct object never worked. Now it raises an error rather than
> silently creating a broken object.
>
> ----------
> nosy: +serhiy.storchaka
> resolution:  -> not a bug
> stage:  -> resolved
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue29628>
> _______________________________________
>
msg288411 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-23 05:59
As a workaround you can make Struct pickleable:

from struct import Struct
import copy, copyreg
def pickle_struct(s):
    return Struct, (s.format,)

copyreg.pickle(Struct, pickle_struct)

s1 = Struct('<i')
s2 = copy.deepcopy(s1)
History
Date User Action Args
2022-04-11 14:58:43adminsetgithub: 73814
2017-02-23 05:59:53serhiy.storchakasetmessages: + msg288411
2017-02-23 05:27:22mesheb82setmessages: + msg288407
2017-02-23 05:22:51serhiy.storchakasetstatus: open -> closed

nosy: + serhiy.storchaka
messages: + msg288406

resolution: not a bug
stage: resolved
2017-02-23 03:55:46mesheb82create