.communicate is a nice API for programs that produce little output, and can be buffered.  While that may cover a wide range of uses, it doesn't cover launching CGI programs, such as is done in http.server.  Now there are nice warnings about that issue in the http.server documentation.

However, while .communicate has the building blocks to provide more general solutions, it doesn't expose them to the user, nor does it separate them into building blocks, rather it is a monolith inside ._communicate.

For example, it would be nice to have an API that would "capture a stream using a thread" which could be used for either stdout or stdin, and is what ._communicate does under the covers for both of them.

It would also be nice to have an API that would "pump a bytestream to .stdin as a background thread.  ._communicate doesn't provide that one, but uses the foreground thread for that.  And, it requires that it be fully buffered.  It would be most useful for http.server if this API could connect a file handle and an optional maximum read count to .stdin, yet do it in a background thread.

That would leave the foreground thread able to process stdout.  It is correct (but not what http.server presently does, but I'll be entering that enhancement request soon) for http.server to read the first line from the CGI program, transform it, add a few more headers, and send that to the browser, and then hook up .stdout to the browser (shutil.copyfileobj can be used for the rest of the stream).  However, there is no deadlock free way of achieving this sort of solution, capturing the stderr to be logged, not needing to buffer a potentially large file upload, and transforming the stdout, with the facilities currently provided by subprocess.  Breaking apart some of the existing building blocks, and adding an additional one for .stdin processing would allow a real http.server implementation, as well as being more general for other complex uses.

You see, for http.server, the stdin