classification
Title: Setting .posix=True for shlex object causes infinite loop in __next__
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Claudiu.Popa, ezio.melotti, roger.serwy, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2012-10-11 19:09 by roger.serwy, last changed 2015-03-10 09:44 by serhiy.storchaka.

Files
File name Uploaded Description Edit
shlex_posix_readonly.patch roger.serwy, 2012-10-11 19:21 review
shlex_posix_property.patch roger.serwy, 2012-10-15 12:25 review
issue16200.patch Claudiu.Popa, 2014-10-14 20:39 review
Messages (6)
msg172677 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-11 19:09
The documentation for shlex does not prohibit the user from setting .posix=True after creating a shlex object. When doing so, the .eof attribute is inconsistent, creating an infinite loop in the __next__ method.

Here's some sample code to recreate the issue:

import shlex
s = shlex.shlex(r"", posix=False)
s.posix = True
list(s)

One possible solution is to make .posix a read-only property. Another is to make .posix a property which sets .eof correctly.
msg172680 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-11 19:21
Attached is a patch to make the .posix property read-only, along with a test. The patch is against 3.4.
msg172953 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-10-15 09:33
Making .posix read-only is technically backward-incompatible, but I'm not sure if there are cases where people might have changed its value without incurring in the bug.  Leaving .posix read/writable and changing .eof accordingly might be a better solution.

As a side note, you can use assertRaises (possibly as a context manager) in the tests.
msg172963 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-15 12:25
The shlex_posix_property.patch makes .posix a read/write property that changes .eof appropriately.
msg229348 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2014-10-14 20:39
Here's a refreshed patch, which applies cleanly on tip. Also, I added a test for the code that generates an infinite loop.
msg237737 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-10 09:44
There are 18 public writable attributes in shlex object, and unthinking setting some of them can make shlex to produce incorrect results or create an infinite loop. For example there is nothing to prevent your from setting the eof attribute. Only 14 of 18 attributes is documented and posix is not in this set.

One of solutions is just does nothing. We are all consenting adults here.

More protective solution is to make undocumented attributes (filestack, posix, pushback, state) private. For backward compatibility we can temporary add properties that will emit deprecation warnings.
History
Date User Action Args
2015-03-10 09:44:23serhiy.storchakasetnosy: + serhiy.storchaka

messages: + msg237737
versions: - Python 2.7
2014-10-14 20:39:41Claudiu.Popasetfiles: + issue16200.patch
versions: + Python 3.5, - Python 3.2, Python 3.3, Python 3.4
nosy: + Claudiu.Popa

messages: + msg229348
2012-10-15 12:25:16roger.serwysetfiles: + shlex_posix_property.patch

messages: + msg172963
2012-10-15 09:33:33ezio.melottisetstage: patch review
messages: + msg172953
versions: + Python 3.2
2012-10-15 03:05:04roger.serwysetnosy: + ezio.melotti
2012-10-11 19:21:55roger.serwysetfiles: + shlex_posix_readonly.patch
keywords: + patch
messages: + msg172680
2012-10-11 19:09:30roger.serwycreate