This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Stanislaw Izaak Pitucha
Recipients Stanislaw Izaak Pitucha
Date 2015-09-10.11:33:41
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1441884822.26.0.222476862534.issue25053@psf.upfronthosting.co.za>
In-reply-to
Content
This is something that happened once and I cannot reproduce anymore. In an IPython session I've done the following (see session below), which resulted in pool.map raising its internal exceptions in cases where it shouldn't. Then it worked again. (see in/out 29)

Running exactly the same lines again did not result in the same behaviour. I tried multiple times in the same session as well as new sessions. Looks like a weird race / corruption.

I'm on Ubuntu's '3.4.3 (default, Mar 26 2015, 22:03:40) \n[GCC 4.9.2]'. Running with 2 virtualised cores (vmware). 64-bit system.

In [21]: import multiprocessing
In [22]: p=multiprocessing.Pool()
... (unrelated, checked p.map? and other helps)
In [26]: class A(object):
    def a(self, i):
        print("boo", i, multiprocessing.current_process())
   ....: 
In [27]: obj = A()
In [28]: p.map(obj.a, [1,2,3,4])
Process ForkPoolWorker-1:
Process ForkPoolWorker-2:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
    task = get()
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
    task = get()
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 357, in get
    return ForkingPickler.loads(res)
AttributeError: Can't get attribute 'A' on <module '__main__'>
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 357, in get
    return ForkingPickler.loads(res)
AttributeError: Can't get attribute 'A' on <module '__main__'>
boo 3 <ForkProcess(ForkPoolWorker-3, started daemon)>
boo 4 <ForkProcess(ForkPoolWorker-3, started daemon)>

^CProcess ForkPoolWorker-4:
Process ForkPoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
    task = get()
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
    task = get()
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 354, in get
    with self._rlock:
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 355, in get
    res = self._reader.recv_bytes()
  File "/usr/lib/python3.4/multiprocessing/synchronize.py", line 96, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 416, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 383, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-28-3be6fbcc359b> in <module>()
----> 1 p.map(obj.a, [1,2,3,4])

/usr/lib/python3.4/multiprocessing/pool.py in map(self, func, iterable, chunksize)
    258         in a list that is returned.
    259         '''
--> 260         return self._map_async(func, iterable, mapstar, chunksize).get()
    261 
    262     def starmap(self, func, iterable, chunksize=None):

/usr/lib/python3.4/multiprocessing/pool.py in get(self, timeout)
    591 
    592     def get(self, timeout=None):
--> 593         self.wait(timeout)
    594         if not self.ready():
    595             raise TimeoutError

/usr/lib/python3.4/multiprocessing/pool.py in wait(self, timeout)
    588 
    589     def wait(self, timeout=None):
--> 590         self._event.wait(timeout)
    591 
    592     def get(self, timeout=None):

/usr/lib/python3.4/threading.py in wait(self, timeout)
    551             signaled = self._flag
    552             if not signaled:
--> 553                 signaled = self._cond.wait(timeout)
    554             return signaled
    555         finally:

/usr/lib/python3.4/threading.py in wait(self, timeout)
    288         try:    # restore state no matter what (e.g., KeyboardInterrupt)
    289             if timeout is None:
--> 290                 waiter.acquire()
    291                 gotit = True
    292             else:

KeyboardInterrupt: 

In [29]: p.map(obj.a, [1,2,3,4])
boo 1 <ForkProcess(ForkPoolWorker-5, started daemon)>
boo 2 <ForkProcess(ForkPoolWorker-6, started daemon)>
boo 3 <ForkProcess(ForkPoolWorker-5, started daemon)>
boo 4 <ForkProcess(ForkPoolWorker-6, started daemon)>
Out[29]: [None, None, None, None]
History
Date User Action Args
2015-09-10 11:33:42Stanislaw Izaak Pituchasetrecipients: + Stanislaw Izaak Pitucha
2015-09-10 11:33:42Stanislaw Izaak Pituchasetmessageid: <1441884822.26.0.222476862534.issue25053@psf.upfronthosting.co.za>
2015-09-10 11:33:42Stanislaw Izaak Pituchalinkissue25053 messages
2015-09-10 11:33:41Stanislaw Izaak Pituchacreate