classification
Title: copy.copy fails on threading.local subclass
Type: behavior Stage:
Components: Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jelle Zijlstra, ned.deily, serhiy.storchaka
Priority: normal Keywords: 3.6regression

Created on 2016-12-14 06:09 by Jelle Zijlstra, last changed 2016-12-15 08:54 by ned.deily.

Files
File name Uploaded Description Edit
thread_local_copy.py Jelle Zijlstra, 2016-12-14 06:09
Messages (4)
msg283165 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2016-12-14 06:09
Calling copy.copy on a threading.local subclass copies attributes over correctly in Python 2.7, but creates an empty object in Python 3.3-3.5 and fails with a pickle-related error in 3.6.

Marking this as a release blocker and assigning to Ned because this appears to be a regression in 3.6. However, given that the behavior in previous Python 3 versions isn't very useful either, I'd personally not want to block 3.6 on it.

I haven't yet looked at code to figure out what is causing the differences in behavior. I couldn't find any tracker issues related to copying or pickling threading.local objects, but may have missed something.

$ cat thread_local_copy.py 
import copy
import threading

class Obj(threading.local):
    def __init__(self):
        self.x = 3

o = Obj()
o2 = copy.copy(o)
assert hasattr(o2, 'x')
$ python2.7 thread_local_copy.py 
$ python3.3 thread_local_copy.py 
Traceback (most recent call last):
  File "thread_local_copy.py", line 10, in <module>
    assert hasattr(o2, 'x')
AssertionError
$ python3.4 thread_local_copy.py 
Traceback (most recent call last):
  File "thread_local_copy.py", line 10, in <module>
    assert hasattr(o2, 'x')
AssertionError
$ python3.5 thread_local_copy.py 
Traceback (most recent call last):
  File "thread_local_copy.py", line 10, in <module>
    assert hasattr(o2, 'x')
AssertionError
$ ./python.exe -V
Python 3.6.0+
$ ./python.exe thread_local_copy.py 
Traceback (most recent call last):
  File "thread_local_copy.py", line 9, in <module>
    o2 = copy.copy(o)
  File "/Users/Jelle/code/cpython/Lib/copy.py", line 96, in copy
    rv = reductor(4)
TypeError: can't pickle Obj objects
msg283167 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2016-12-14 06:26
This might be due to issue22995.
msg283169 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-14 07:11
copy.copy() didn't work correctly with threading.local in 3.x. It just silently created an empty instance (even not properly initialized, since __init__ is bypassed). Now it correctly raises TypeError.

copy.copy() looks working in 2.7 because it copied the instance __dict__. But the internal state can be lost. There are no tests.

Definitely this is not 3.6 regression. And it is questionable that it is 3.x regression, because it is questionable that copying worked in 2.x.

I would consider making threading.local copyable a new feature. But it is questionable that this feature is useful.
msg283260 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-12-15 08:54
Thanks for raising the issue, Jelle.  And thanks for the analysis, Serhiy.  It doesn't sound like a release blocker for 3.6.0.  Is it worth addressing at all?
History
Date User Action Args
2016-12-15 08:54:54ned.deilysetpriority: release blocker -> normal
assignee: ned.deily ->
messages: + msg283260

versions: + Python 3.7
2016-12-14 07:11:42serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg283169
2016-12-14 06:26:46Jelle Zijlstrasetmessages: + msg283167
2016-12-14 06:09:48Jelle Zijlstracreate