The attached snippet causes a deadlock just about every time it's run (tested with 3.6.7/Ubuntu, 3.7.1/Arch, 3.6.7/OSX, and 3.7.1/OSX -- deadlock seems to be less frequent on the last, but still common). The issue appears to be something like the following sequence of events:

1. The main thread calls pool.__exit__, eventually entering Pool._terminate_pool.
2. result_handler's state is set to TERMINATE, causing it to stop reading from outqueue.
3. The main thread, in _terminate_pool, joins on worker_handler, which is (usually) in the middle of sleeping for 0.1 seconds, opening a window for the next two steps to occur.
4. The worker process finishes its task and acquires the shared outqueue._wlock.
5. The worker attempts to put the result into outqueue, but its pickled form is too big to fit into the buffer of os.pipe, and it blocks here with the lock held.
6. worker_handler wakes up and exits, freeing _terminate_pool to continue.
7. _terminate_pool terminates the worker.
8. task_handler tries to put None into outqueue, but blocks, since the lock was acquired by the terminated worker.
9. _terminate_pool joins on task_handler, and everything is deadlocked.

Hello, dzhu. I can reproduce on my OSX, since you already dive into the code, do you have any idea to fix or improve it?

I will work on it if no one wants to create a PR for this next week.

Given the hairiness of the deadlock, I think I would rather let someone who has more experience with the codebase in general handle it, but I can come back to it if it doesn't get addressed.

This is a great example of abusing the multi-processing API and thus
creating timing errors that lead to locks not being released. What is
happening is that the example attempts to transmit data that is too
big for the underlying pipe and this creates the timing errors since
the pipe that returns the result of invoking apply_async does not have
the capacity of transmitting the result of the worker process in a single
operation and accordingly the worker process returns prematurely while
still holding the pipe lock. This can be seen by testing the ready status
of the result returned by apply_async which will show whether the complete
result of the worker process has been received.
  
 The solution to this situation is simple, simply invoke get() on
 the asynchronous result returned by apply_async. Using this call has the
 effect of correctly synchronizing the pipe used by the low-level queues
 used by the Pool Manager and there will be no lock left active when it
 should'nt be. The script lock1.py contains code that implements this fix
 as well as verifying that everything has worked properly. The output from
 the script is found in the file lock1.result.txt.
  
 Because there is an API based solution to the problem plus the
 behavior of apply_async makes sense in that the process of transferring
 multi-buffer data is very CPU intensive and should be delegated to the
 worker process rather than to the main-line process, I do not recommend that
 any changes be made to the multiprocessing code in Python, rather the
 solution should use the available multiprocessing API correctly.

As Jonathan Gossage said, I think it may break some code to fix this issue, maybe we could just add a warning on the document?

I think documentation is sufficient but I would like it to state the pitfalls available if apply_async is not synchronized correctly which will happen whenever the output does not fit the pipe buffer.

I second what @vstinner already said in the comments for PR11143, that this should not merely be documented.