classification
Title: pipes seems designed for bytes but is str-only
Type: Stage:
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, rtollert, terry.reedy
Priority: normal Keywords:

Created on 2021-09-05 18:54 by rtollert, last changed 2021-09-10 23:26 by gvanrossum.

Messages (3)
msg401103 - (view) Author: Richard Tollerton (rtollert) Date: 2021-09-05 18:54
1. https://github.com/python/cpython/blob/3.9/Lib/pipes.py#L6

> Suppose you have some data that you want to convert to another format,
> such as from GIF image format to PPM image format.

2. https://docs.python.org/3.9/library/pipes.html

> Because the module uses /bin/sh command lines, a POSIX or compatible shell for os.system() and os.popen() is required.

3. https://docs.python.org/3.9/library/os.html#os.popen

> The returned file object reads or writes text strings rather than bytes.


(1) and (3) are AFAIK mutually contradictory: you can't reasonably expect to shove GIFs down a str file object. I'm guessing that pipes is an API that never got its bytes API fleshed out?

My main interest in this is that I'm writing a large CSV to disk and wanted to pipe it through zstd first. And I wanted something like perl's open FILE, "|zstd -T0 -19 > out.txt.zst". But the CSV at present is all bytes. (Technically the content is all latin1 at the moment, so I may have a workaround, but I'm not 100% certain it will stay that way.)

What I'd like to see is for pipes.Template.open() to accept 'b' in flags, and for that to be handled in the usual way.
msg401617 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-09-10 22:33
guido, you seem to be the only active contributor to the pipes module.  What do you think?
msg401619 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-09-10 23:26
I think that module is probably a relic from the Python 1 days…
History
Date User Action Args
2021-09-10 23:26:41gvanrossumsetmessages: + msg401619
2021-09-10 22:33:22terry.reedysetnosy: + gvanrossum, terry.reedy
messages: + msg401617
2021-09-05 18:54:59rtollertcreate