New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
imap from ThreadPool hangs by an exception in a generator function #72882
Comments
It's only in imap, in map it's ok. The following code explains the issue: from multiprocessing.pool import ThreadPool
pool = ThreadPool(10)
def gen():
yield 1 + '1' # here is an error try: |
Using the supplied example, I was unable to reproduce what you described using either 3.5 or 3.6-beta on OS X 10.11. What platform are you using? (Perhaps it is platform specific.) |
Ubuntu 14.04 LTS, 3.13.0-83-generic, x86_64 |
So, I've checked twice, it's presented by me on python 3.4.3, and not by 3.5.2. So I will go deaper |
I've reproduced it on 2 different machines:
|
It's hanging in a while loop in _handle_workers, /usr/lib/python3.4/multiprocessingpool.py:365. @staticmethod
def _handle_workers(pool):
thread = threading.current_thread()
# Keep maintaining workers until the cache gets drained, unless the pool
# is terminated.
while thread._state == RUN or (pool._cache and thread._state != TERMINATE):
pool._maintain_pool()
time.sleep(0.1)
# send sentinel to stop workers
pool._taskqueue.put(None)
util.debug('worker handler exiting') |
If it only occurs in 3.4.x, is moving to 3.5 an option for you? |
Yes, I'm free to move to 3.5, now I'm seeing isn't there any problems in 3.5 according to this issue. |
In Py3.6, it raises error: >>> next((pool.imap(str, gen())))
Traceback (most recent call last):
File "/opt/lib/python3.7/multiprocessing/pool.py", line 684, in next
item = self._items.popleft()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/lib/python3.7/multiprocessing/pool.py", line 690, in next
item = self._items.popleft()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/lib/python3.7/multiprocessing/pool.py", line 693, in next
raise StopIteration
StopIteration But this doesn't looks good. I think it should raise TypeError. It can be fixed in bpo-28699. |
Closing this issue -- all further discussion moves to bpo-28699 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: