This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: shlex.quote applied to a glob pattern disables glob matching
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.smith, geophile, remi.lapeyre
Priority: normal Keywords:

Created on 2020-04-22 15:21 by geophile, last changed 2022-04-11 14:59 by admin.

Messages (4)
msg367010 - (view) Author: Jack Orenstein (geophile) Date: 2020-04-22 15:21
I am using shlex.quote to pass filenames to a shell command, e.g. ls. The problem is that glob patterns, when quoted, no longer operate as glob patterns. So, for example, executing this:

    ls 'var/log/syslog*'

results in this output:

    ls: cannot access '/var/log/syslog*': No such file or directory

The documentation for shlex.quote says: "Return a shell-escaped version of the string s. The returned value is a string that can safely be used as one token in a shell command line, for cases where you cannot use a list."

I believe that quoting needs to preserve glob semantics to fulfill the documented goal. For example, it seems to me that quoting as follows would be better:

    ls '/var/log/syslog'*
msg367011 - (view) Author: Rémi Lapeyre (remi.lapeyre) * Date: 2020-04-22 15:39
shlex.quote makes the string safe to pass a command, what if it's 

   rm 'var/log/syslog*'

instead?

You make sure that only the file given would be removed but then shlex.quote() shoot you in the foot.

This would also cause issues for files with '*' or another special characters in the name, you would not be able to pass their name anymore.

Also, not all shells have the same glob patterns and some of them are actually configurable to enable more patterns, so it would be impossible to know what to escape or not, shlex.quote() just quote everything unconditionnaly

If you want to allow '*' at the end or inside the pattern I think the best way is to look for it in your application, split (or take the prefix if you only want to allow it in the end), use shlex.quote() on the parts and concatenate with '*'.
msg367038 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-04-22 18:22
Agreed that shlex.quote is working as intended. It's goal is to pass something to the shell, preserving it just as you started with it. It exists exactly because you want to avoid shell globbing and parsing issues (consider filenames that contain spaces and *'s).
msg367039 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-04-22 18:24
I meant to add: it's possible that this isn't well documented.
History
Date User Action Args
2022-04-11 14:59:29adminsetgithub: 84543
2020-04-22 18:24:04eric.smithsetmessages: + msg367039
2020-04-22 18:22:06eric.smithsetnosy: + eric.smith
messages: + msg367038
2020-04-22 15:40:21remi.lapeyresettype: behavior
versions: + Python 3.9, - Python 3.7
2020-04-22 15:39:56remi.lapeyresetnosy: + remi.lapeyre
messages: + msg367011
2020-04-22 15:21:31geophilecreate