This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: shlex.shlex with punctuation_chars and posix doesn't handle punctuation next to quotes
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: evan_, python-dev, r.david.murray, vinay.sajip
Priority: normal Keywords: patch

Created on 2017-01-02 12:56 by evan_, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
shlex-posix-quote.diff evan_, 2017-01-02 15:38 Naive fix for issue29132 review
shlex-posix-quote2.diff evan_, 2017-01-06 11:39 review
Messages (5)
msg284480 - (view) Author: Evan Andrews (evan_) * Date: 2017-01-02 12:56
When both punctuation_chars and posix are used, any punctuation directly next to quoted characters is interpreted as a single token.

>>> from shlex import shlex
>>> list(shlex('>"a"', posix=True))
['>', 'a']
>>> list(shlex('>"a"', punctuation_chars=True))
['>', '"a"']
>>> list(shlex('>"a"', posix=True, punctuation_chars=True))
['>a']  # should be ['>', 'a']
msg284484 - (view) Author: Evan Andrews (evan_) * Date: 2017-01-02 15:38
I've attached a simplistic patch which fixes the problem. I suspect a better fix might be to shuffle the elif branches around so the extra condition isn't necessary. Vinay, what are your thoughts?
msg284763 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2017-01-05 17:15
Evan, thanks for looking at this. Would you like to submit a completed contributor form?

https://www.python.org/psf/contrib/contrib-form/
msg284814 - (view) Author: Evan Andrews (evan_) * Date: 2017-01-06 11:39
I've just submitted the form.

I'm attaching a second patch which also addresses another similar bug demonstrated in the test case. The fix is to check for the 'c' state before checking for quotes or escape characters.
msg285509 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-01-15 10:08
New changeset 7e40c940df14 by Vinay Sajip in branch '3.6':
Fixed #29132: Updated shlex to work better with punctuation chars in POSIX mode.
https://hg.python.org/cpython/rev/7e40c940df14

New changeset c163a76163f7 by Vinay Sajip in branch 'default':
Closes #29132: Merged fix from 3.6.
https://hg.python.org/cpython/rev/c163a76163f7
History
Date User Action Args
2022-04-11 14:58:41adminsetgithub: 73318
2017-01-15 10:08:05python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg285509

resolution: fixed
stage: resolved
2017-01-06 11:39:26evan_setfiles: + shlex-posix-quote2.diff

messages: + msg284814
2017-01-05 17:15:02vinay.sajipsetmessages: + msg284763
2017-01-02 15:38:07evan_setfiles: + shlex-posix-quote.diff
keywords: + patch
messages: + msg284484
2017-01-02 12:56:23evan_create