What is the full version of Python you used? If this is not 3.6.3 does upgrading to 3.6.3 help? Try also with 3.6.4rc1 and the beta of 3.7.

Do you use threads, forking, signal handlers? Is it synchronous or asynchronous code? Could you show the code of the function decorated with the contextmanager (or asynccontextmanager) decorator and the code that uses this context manager?

With such little information we can fix a bug only by accident, when fixing other bugs or during refactoring. And there is a chance that it is already fixed in 3.6.3, 3.6.4rc1 or 3.7.