classification
Title: copy breaks staticmethod
Type: behavior Stage: resolved
Components: Interpreter Core, Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: benjamin.peterson Nosy List: alexandre.vassalotti, barry, benjamin.peterson, dangyogi, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-03-25 13:16 by dangyogi, last changed 2017-08-02 08:34 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 925 closed serhiy.storchaka, 2017-03-31 10:26
PR 2823 merged serhiy.storchaka, 2017-07-23 10:53
Messages (12)
msg290481 - (view) Author: Bruce Frederiksen (dangyogi) Date: 2017-03-25 13:16
Doing a copy on a staticmethod breaks it:

Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from copy import copy
>>> def foo(): pass
... 
>>> class bar: pass
... 
>>> bar.x = staticmethod(foo)
>>> bar.x.__name__
'foo'
>>> bar.y = copy(staticmethod(foo))
>>> bar.y.__name__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: uninitialized staticmethod object
msg290484 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-25 14:02
Copying and pickling of staticmethod objects are not supported. Starting from 3.6 this raises an exception (see issue22995).

>>> bar.y = copy(staticmethod(foo))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython3.6/Lib/copy.py", line 96, in copy
    rv = reductor(4)
TypeError: can't pickle staticmethod objects
msg290905 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-31 10:31
This issue was fixed by issue22995 in 3.6. But for some reasons the general solution was not applied to 3.5 and 2.7. Proposed patch makes staticmethod, classmethod and property descriptors explicitly non-pickleable (as was made explicitly non-pickleable file-like object) in 3.5. Only tests will be foreported to 3.6 and 3.7.
msg291739 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-16 06:34
What are your thoughts Barry? staticmethod, classmethod and property objects are incorrectly pickled in 2.7 and 3.5 (they are not pickleable in 3.6+). Making them not pickleable in 2.7 and 3.5 can help to catch programming errors.

But this can break the code that "just works" (actually works incorrectly) if it pickles or deepcopies complex objects containing staticmethod, classmethod or property objects, but never use it. Similar situation happened in issue22995 with Cython objects.

Maybe emitting a warning rather than an exception would be safer? But deprecation warnings are ignored by default.
msg291748 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-04-16 15:12
I'd be very hesitant to add anything to 2.7 that changes (even broken) behavior here.  It might make more sense to backport the more strict checks to 3.5.  OTOH, we can save people from all programming errors, and if warnings are basically ignored (plus, adding warnings *can* break things), then perhaps we should only document the limitations.

There's already some description of copy()'s limitations, so either add another warning there, or in the static method objects description in section 3.2
msg291752 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-16 16:42
If add just warnings we are free to add any warnings in 2.7 in Py3k compatible mode (option -3).
msg298892 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-23 07:00
I have decided to not merge these changes in 3.5. They help to avoid programmatic errors, but have a non-zero chance of breaking programs that work by accidence. Since this is the last non-secure bugfix we will not have a chance to rollback these changes.

An alternative to these changes is testing a program with Python 3.6.
msg298896 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-23 10:56
But I think it is worth to add a check and emit a deprecation warning in Py3k mode in 2.7. 2.7 has longer support term than 3.5. And since it is less compatible with 3.6, it is harder to use 3.6 for testing 2.7 programs.
msg299607 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-08-01 06:41
Thank you Benjamin for your review!

But I have a design question. Is DeprecationWarning a correct warning type? DeprecationWarning is used when some feature less or more works in the current version, but will be removed in future releases. But pickling memoryview and staticmethod don't work and never worked correctly. A warning here is a sign that the code can not work as expected. Wouldn't RuntimeWarning be more appropriate warning type?
msg299642 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2017-08-02 05:00
I don't think it's too important, though most py3k warnings have been
DeprecationWarning. You can think of it as deprecating a misfeature.
msg299648 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-08-02 08:33
New changeset 3dd1ccbb0950b2b83713a495958c35d60b453fa9 by Serhiy Storchaka in branch '2.7':
bpo-29902: Emit a Py3k deprecation warning when pickling or copying (#2823)
https://github.com/python/cpython/commit/3dd1ccbb0950b2b83713a495958c35d60b453fa9
msg299649 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-08-02 08:34
Thank you Benjamin.
History
Date User Action Args
2017-08-02 08:34:21serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg299649

stage: patch review -> resolved
2017-08-02 08:33:36serhiy.storchakasetmessages: + msg299648
2017-08-02 05:00:57benjamin.petersonsetmessages: + msg299642
2017-08-01 06:41:53serhiy.storchakasetmessages: + msg299607
2017-07-27 10:28:02serhiy.storchakasetassignee: benjamin.peterson

nosy: + benjamin.peterson
2017-07-23 10:56:44serhiy.storchakasetmessages: + msg298896
stage: patch review
2017-07-23 10:53:28serhiy.storchakasetpull_requests: + pull_request2875
2017-07-23 07:00:14serhiy.storchakasetmessages: + msg298892
versions: - Python 3.5
2017-04-16 16:42:32serhiy.storchakasetmessages: + msg291752
2017-04-16 15:12:48barrysetmessages: + msg291748
2017-04-16 06:34:24serhiy.storchakasetmessages: + msg291739
2017-03-31 10:31:09serhiy.storchakasetmessages: + msg290905
2017-03-31 10:26:04serhiy.storchakasetpull_requests: + pull_request822
2017-03-25 14:02:29serhiy.storchakasetversions: + Python 2.7
nosy: + barry, alexandre.vassalotti, serhiy.storchaka

messages: + msg290484

components: + Interpreter Core
2017-03-25 13:16:44dangyogicreate