The newly added shlex.punctuation_chars is special compared to the other public instance variables: It can ONLY be used when constructing a shlex instance, unlike other public instance variables, such as commenters, which can ONLY be set later.

>>> s = shlex.shlex('abc // def')
>>> s.commenters = '/'
>>> list(s)
['abc', '', '']

>>> s = shlex.shlex('abc // def', punctuation_chars = '/')
>>> list(s)
['abc', '//', 'def']

However, setting punctuation_chars later shows this rather useless error message:

>>> s = shlex.shlex('abc // def')
>>> s.punctuation_chars = '/'
>>> list(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/python/3.7.1/lib/python3.7/shlex.py", line 295, in __next__
    token = self.get_token()
  File "/opt/python/3.7.1/lib/python3.7/shlex.py", line 105, in get_token
    raw = self.read_token()
  File "/opt/python/3.7.1/lib/python3.7/shlex.py", line 133, in read_token
    if self.punctuation_chars and self._pushback_chars:
AttributeError: 'shlex' object has no attribute '_pushback_chars'

Thanks for the report. The code was added with c1f974c944a3e73cbc9102356d8700a190dcafb3 and self._pushback_chars is declared only when punctuation_chars is passed to shlex.shlex in the constructor as you have mentioned in https://github.com/python/cpython/blob/master/Lib/shlex.py#L59 . Adding Vinay for thoughts on the usage.

I agree that it's inconsistent, but quite a bit of setting up is done when punctuation_chars is provided, as per the link in msg329312. One could convert the attribute to a property and have a setter that does the equivalent set up, but some of the setup is one-time (removing things from wordchars, etc.), and would require additional work to handle the case where the property is reassigned multiple times. I have no problem updating the documentation to indicate in a note that it must be provided in the constructor and not later, but apart from the fact that it's inconsistent, is there a use case for supporting setting it later? That would mean setting it up so that you could set it several times, unset it, etc.

It makes sense to me that information used in an expensive one-time setup should be specified in advance where other parameters that are more easily changed are specified downstream.  The API reflects the a sensible way to use the tool.  Making it to easy to change later increases the risk of misuse.

So a documentation update and a better run time error message which clarifies that shlex.punctuation_chars is read-only?

> a better run time error message which clarifies that shlex.punctuation_chars is read-only

That it can be set only via the __init__(), yes.

New changeset 972cf5c06a5ba16ad243a442dbb9c15307fbed95 by Vinay Sajip (Alex) in branch 'master':
bpo-35168: Make shlex.punctuation_chars read-only (#11631)
https://github.com/python/cpython/commit/972cf5c06a5ba16ad243a442dbb9c15307fbed95

New changeset aca878ecf18b9e915e9b551f095a550b5c6e9bc5 by Vinay Sajip in branch '3.7':
[3.7] bpo-35168: Make shlex.punctuation_chars read-only (GH-11631) (GH-15926)
https://github.com/python/cpython/commit/aca878ecf18b9e915e9b551f095a550b5c6e9bc5

New changeset 3b92ddb7612fd8fe2be95ff3d8022ed18335b13f by Vinay Sajip in branch '3.8':
[3.8] bpo-35168: Make shlex.punctuation_chars read-only (GH-11631) (GH-15927)
https://github.com/python/cpython/commit/3b92ddb7612fd8fe2be95ff3d8022ed18335b13f