I was looking to implement multiprocess pool. It worked fine with the numpy function while with the user defined function it ran into error.
import numpy
>>> import multiprocessing
>>> P = multiprocessing.Pool(5)
>>> P.map(numpy.sqrt,range(50))
[0.0, 1.0, 1.4142135623730951, 1.7320508075688772, 2.0, 2.2360679774997898, 2.4494897427831779, 2.6457513110645907, 2.8284271247461903, 3.0, 3.1622776601683795, 3.3166247903553998, 3.4641016151377544, 3.6055512754639891, 3.7416573867739413, 3.872983346207417, 4.0, 4.1231056256176606, 4.2426406871192848, 4.358898943540674, 4.4721359549995796, 4.5825756949558398, 4.6904157598234297, 4.7958315233127191, 4.8989794855663558, 5.0, 5.0990195135927845, 5.196152422706632, 5.2915026221291814, 5.3851648071345037, 5.4772255750516612, 5.5677643628300215, 5.6568542494923806, 5.7445626465380286, 5.8309518948453007, 5.9160797830996161, 6.0, 6.0827625302982193, 6.164414002968976, 6.2449979983983983, 6.324555320336759, 6.4031242374328485, 6.4807406984078604, 6.5574385243020004, 6.6332495807107996, 6.7082039324993694, 6.7823299831252681, 6.8556546004010439, 6.9282032302755088, 7.0]
>>> def f(x):
return x*x
>>> P.map(f, range(50))
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 530, in __bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 483, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Python27\lib\multiprocessing\pool.py", line 285, in _handle_tasks
put(task)
TypeError: expected string or Unicode object, NoneType found
|
Yes actually it produce no erroe on standalone script. But the script
executes without any outputs. not even for numpy function. I used code
below in script.
import multiprocessing
import numpy
def f(x):
return x*x
if __name__ = "__main__":
p= multiprocessing.Pool(5)
print p.map(numpy.sqrt,[1,2,3,4])
print p.map(f,[1,2,3,4])
On May 15, 2015 2:52 PM, "Paul Moore" <report@bugs.python.org> wrote:
>
> Paul Moore added the comment:
>
> Multiprocessing works by firing up additional processes. Those processes
> won't have access to functions defined in the interactive interpreter.
>
> Can you reproduce this problem in a standalone script? I suspect not, but
> if you can please post the script here.
>
> Marking as "not a bug", but I haven't closed it yet in case a script
> reproducing the issue can be provided.
>
> ----------
> nosy: +paul.moore
> resolution: -> not a bug
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue24202>
> _______________________________________
>
|
OK, if it's not reproducible in a standalone script, I'll close this as it's expected behaviour.
Correcting the typo in your script (__name__ == "__main__"), I ran it and it worked as expected on my system:
>type multi.py
import multiprocessing
import numpy
def f(x):
return x*x
if __name__ == "__main__":
p= multiprocessing.Pool(5)
print(p.map(numpy.sqrt,[1,2,3,4]))
print(p.map(f,[1,2,3,4]))
PS 15:05 {00:00.089} C:\Work\Scratch
>py .\multi.py
[1.0, 1.4142135623730951, 1.7320508075688772, 2.0]
[1, 4, 9, 16]
I'm not sure why you weren't getting output, but it doesn't look like a Python issue.
|