New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
subprocess: more general (non-buffering) communication #42281
Comments
Right now you can use subprocess.Popen.communicate() to The functionality of communicate() (which is somewhat I attached a function I'm using which does that for the If patch 1175984 was accepted, then this wouldn't be |
Logged In: YES Would an asynchronous subprocess (which you would poll |
Logged In: YES Also, what you post is not a 'bug', it is a 'feature request'. |
Logged In: YES Reclassifying as RFE. Assigning to Peter. |
Closed bpo-14872 as a duplicate of this. |
Please note the work being performed on bpo-1191964 which was first referenced in msg54591. |
The non-blocking read and write features proposed in bpo-1191964 wouldn’t be sufficient. Perhaps they might be useful for implementing this feature on Windows though; I’m not sure. And a non-blocking write on Posix may be slightly more efficient as well, removing the limitation of copying in chunks of PIPE_BUF bytes. Ian’s original patch posted here proposes a version of communicate() which accepts a file reader for the subprocess’s input, and file writers for the subprocess’s outputs. Another option could be something like my SubprocessWriter class <https://github.com/vadmium/pacman-tools/blob/9ffdd88/makeaur#L179\>. It provides a file writer interface for the caller to write to the subprocess’s input, while copying data from the subprocess’s output behind the scenes. I used it to stream a tar file, as it is written by the “tarfile” module, into a GPG subprocess to create a digital signature, while writing the output of GPG into a BytesIO buffer. Using Unix command pipeline pseudocode, it is used a bit like this: tarfile.open(mode="w|") | Popen("gpg") | BytesIO() |
since the time this was filed, subprocess has evolved a lot and third party options for child process have appeared as well as modern things like: https://docs.python.org/3/library/asyncio-subprocess.html (stdlib) https://trio.readthedocs.io/en/stable/reference-io.html?highlight=subprocess#options-for-starting-subprocesses (trio, easier async framework to use than asyncio) plumbing stdin/stdout/stderr data to python file-like objects from the stdlib subprocess APIs themselves is better left to third party solutions building on top of the subprocess module today. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: