New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
shlex not posix compliant when parsing "foo#bar" #51860
Comments
The shlex parser parses "foo#bar" as "foo", discarding the rest as a However, POSIX (see below) only allows comments to start at the Fixing this might break some applications that rely on this broken POSIX 2008, Rationale C.2.3 (which refers to Shell & Utilities 2.3(10)): The (10) rule about '#' as the current character is the first in the |
Attached a program which shows the relevant behaviour: import shlex
tests = [ "foo#bar", "foo #bar" ]
for t in tests:
print "%s -> %s" % (t,[x for x in shlex.shlex(t,posix=True)]) results in $ python lexer_test.py
foo#bar -> ['foo']
foo #bar -> ['foo'] (expected of course is ['foo#bar'] on the first line). |
Here's a patch addressing the behavior described. |
Given that test_shlex.py tests for the current behavior, it is hard to call this a bug in the tracker sense of the term. I would only change with a new version. The manual just says "When operating in POSIX mode, shlex will try to be as close as possible to the POSIX shell parsing rules." but gives no reference to which authority it is following or what the rules are in either case. Manual section 23.2.2. Parsing Rules only discusses the differences between posix and non-posix rules, not the common rules. I suspect this module was written well over a decade ago, maybe closer to two. Is it possible that earlier versions were different on this issue? Or is the 2008 version only cosmetically different some 1990s version? |
See also bpo-1521950. |
The doc section has no reference, as in a live web link, to any version of the POSIX specification. This is unlike other doc sections that implement various RFCs (which also get updated). The docs also link to specific references for the Unicode version supported, which has changed from version to version. The OP quotes (without giving a link) from the 2008 version. POSIX and shlex are much older than that, implying that shlex might conform to an earlier version, just as other modules implement older RFCs that have been superceded. |
Here a some of the relevant links from POSIX 2008:
Sections 2.3 (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_03) and 2.10 (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_10) of [1] are particularly relevant. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: