Message331260
This is a great example of abusing the multi-processing API and thus
creating timing errors that lead to locks not being released. What is
happening is that the example attempts to transmit data that is too
big for the underlying pipe and this creates the timing errors since
the pipe that returns the result of invoking apply_async does not have
the capacity of transmitting the result of the worker process in a single
operation and accordingly the worker process returns prematurely while
still holding the pipe lock. This can be seen by testing the ready status
of the result returned by apply_async which will show whether the complete
result of the worker process has been received.
The solution to this situation is simple, simply invoke get() on
the asynchronous result returned by apply_async. Using this call has the
effect of correctly synchronizing the pipe used by the low-level queues
used by the Pool Manager and there will be no lock left active when it
should'nt be. The script lock1.py contains code that implements this fix
as well as verifying that everything has worked properly. The output from
the script is found in the file lock1.result.txt.
Because there is an API based solution to the problem plus the
behavior of apply_async makes sense in that the process of transferring
multi-buffer data is very CPU intensive and should be delegated to the
worker process rather than to the main-line process, I do not recommend that
any changes be made to the multiprocessing code in Python, rather the
solution should use the available multiprocessing API correctly. |
|
Date |
User |
Action |
Args |
2018-12-07 01:44:10 | Jonathan.Gossage | set | recipients:
+ Jonathan.Gossage, pitrou, Windson Yang, dzhu, nconway |
2018-12-07 01:44:10 | Jonathan.Gossage | set | messageid: <1544147050.65.0.788709270274.issue35267@psf.upfronthosting.co.za> |
2018-12-07 01:44:10 | Jonathan.Gossage | link | issue35267 messages |
2018-12-07 01:44:09 | Jonathan.Gossage | create | |
|