Message264608
I'm using Python 3.5.1 x86-64 on Windows Server 2008 R2. Trying to run the ProcessPoolExecutor example [1] generates this exception:
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Program Files\Python35\lib\threading.py", line 914, in _bootstrap_inner
self.run()
File "C:\Program Files\Python35\lib\threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "C:\Program Files\Python35\lib\concurrent\futures\process.py", line 270, in _queue_management_worker
ready = wait([reader] + sentinels)
File "C:\Program Files\Python35\lib\multiprocessing\connection.py", line 859, in wait
ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)
File "C:\Program Files\Python35\lib\multiprocessing\connection.py", line 791, in _exhaustive_wait
res = _winapi.WaitForMultipleObjects(L, False, timeout)
ValueError: need at most 63 handles, got a sequence of length 64
The problem seems to be related to the value of the Windows constant MAXIMUM_WAIT_OBJECTS (see [2]), which is 64. This machine has 64 logical cores, so ProcessPoolExecutor defaults to 64 workers.
Lowering max_workers to 63 or 62 still results in the same exception, but max_workers=61 works fine.
[1] https://docs.python.org/3.5/library/concurrent.futures.html#processpoolexecutor-example
[2] https://hg.python.org/cpython/file/80d1faa9735d/Modules/_winapi.c#l1339 |
|
Date |
User |
Action |
Args |
2016-05-01 20:45:38 | diogocp | set | recipients:
+ diogocp, paul.moore, tim.golden, zach.ware, steve.dower |
2016-05-01 20:45:38 | diogocp | set | messageid: <1462135538.41.0.857077084248.issue26903@psf.upfronthosting.co.za> |
2016-05-01 20:45:38 | diogocp | link | issue26903 messages |
2016-05-01 20:45:37 | diogocp | create | |
|