The non-blocking read and write features proposed in Issue 1191964 wouldn’t be sufficient. Perhaps they might be useful for implementing this feature on Windows though; I’m not sure. And a non-blocking write on Posix may be slightly more efficient as well, removing the limitation of copying in chunks of PIPE_BUF bytes.

Ian’s original patch posted here proposes a version of communicate() which accepts a file reader for the subprocess’s input, and file writers for the subprocess’s outputs. Another option could be something like my SubprocessWriter class <https://github.com/vadmium/pacman-tools/blob/9ffdd88/makeaur#L179>. It provides a file writer interface for the caller to write to the subprocess’s input, while copying data from the subprocess’s output behind the scenes. I used it to stream a tar file, as it is written by the “tarfile” module, into a GPG subprocess to create a digital signature, while writing the output of GPG into a BytesIO buffer. Using Unix command pipeline pseudocode, it is used a bit like this:

tarfile.open(mode="w|") | Popen("gpg") | BytesIO()