classification
Title: "Can't pickle local object" when uses functools.partial with method and args...
Type: behavior Stage:
Components: asyncio Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: DAVID ALEJANDRO Pineda, serhiy.storchaka, yselivanov
Priority: normal Keywords:

Created on 2017-02-09 21:41 by DAVID ALEJANDRO Pineda, last changed 2017-02-15 21:28 by gvanrossum.

Files
File name Uploaded Description Edit
code_with_bug.png DAVID ALEJANDRO Pineda, 2017-02-09 21:41 Code with bug
Messages (2)
msg287459 - (view) Author: DAVID ALEJANDRO Pineda (DAVID ALEJANDRO Pineda) * Date: 2017-02-09 21:41
Hello.

I'm working in a real time data collector project. I'm using asyncio and multiprocessing (run_in_executor).
In python 3.5 worked fine but in python 3.6 not, gave to me this error:
 "Traceback (most recent call last):
  File "/usr/local/lib/python3.6/multiprocessing/queues.py", line 241, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/local/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'WeakSet.__init__.<locals>._remove'
"

I tracked and the problem is when i run some method in run_in_executor with the functools.partial.

Before this version the system works fine.

The main file is local.py (to run with local administration using a socket)
And the engine is in engine.py

The project (in development):
https://gitlab.com/pineiden/collector
msg287887 - (view) Author: DAVID ALEJANDRO Pineda (DAVID ALEJANDRO Pineda) * Date: 2017-02-15 21:18
Now
I try again with different ways to run a method with args, without args works fine to insert in the executor.
I find there are big differences with file functools betwen versions 3.5.1 to 3.6.0.
 Here a more simple example for test the functools:
https://github.com/dpineiden/async_multiprocessing
History
Date User Action Args
2017-02-15 21:28:40gvanrossumsetnosy: - gvanrossum
2017-02-15 21:18:46DAVID ALEJANDRO Pinedasetmessages: + msg287887
2017-02-10 01:15:51r.david.murraysetnosy: + serhiy.storchaka
2017-02-09 21:41:25DAVID ALEJANDRO Pinedacreate