Message245509
See:
http://stackoverflow.com/questions/30943161/multiprocessing-pool-with-maxtasksperchild-produces-equal-pids
The documentation never makes clear what a "task" in the context of Pool.map. At best, it says:
"This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer."
in the map documentation. However it does not say how this chunks are calculated by default, making the maxtasksperchild argument not very useful. The fact that a function evaluated by map is not a "task" should be much clearer in the documentation.
Also, in the examples, such as:
with multiprocessing.Pool(PROCESSES) as pool:
#
# Tests
#
TASKS = [(mul, (i, 7)) for i in range(10)] + \
[(plus, (i, 8)) for i in range(10)]
results = [pool.apply_async(calculate, t) for t in TASKS]
imap_it = pool.imap(calculatestar, TASKS)
imap_unordered_it = pool.imap_unordered(calculatestar, TASKS)
TASKS are not actually "tasks" but rather "task groups". |
|
Date |
User |
Action |
Args |
2015-06-19 17:05:37 | Zahari.Dim | set | recipients:
+ Zahari.Dim, docs@python |
2015-06-19 17:05:37 | Zahari.Dim | set | messageid: <1434733537.74.0.146405679971.issue24475@psf.upfronthosting.co.za> |
2015-06-19 17:05:37 | Zahari.Dim | link | issue24475 messages |
2015-06-19 17:05:37 | Zahari.Dim | create | |
|