See the example below (also attached).

First example: The lineno reported just after "word2" is pulled is 2.
Second example: The lineno reported just after "," is pulled is still 1.

This behaviour seems inconsistent. The lineno should increment either when the last token of a line is pulled, or after the first token from the next line (in my opinion preferably the former). It should not have different bahaviour depending on what type of token that is (alpha vs. comma).

I have repeated this on 

Also, does Issue 16121 relate to this?


#!/usr/bin/env python
import shlex

first = shlex.shlex("word1 word2\nword3")
print (first.get_token())
print (first.get_token())
print ("line no", first.lineno)
print ("")

second = shlex.shlex("word1 word2,\nword3")
print (second.get_token())
print (second.get_token())
print (second.get_token())
print ("line no", second.lineno)


# Output:
# word1
# word2
# line no 2
#
# word1
# word2
# ,
# line no 1

From the unfinished sentence:

I have repeated this on all versions of shlex on which I have tried. Including Python 2.6, 2.7, 3.2 and 3.3.

There was a parameter `punctuation_chars` added to the shlex.shlex class with issue 1521950 (implemented for 3.6).  Although the comma is not one of the default punctuation characters (setting the parameter to punctuation_chars=True won't change the behavior), you can use `punctuation_chars=","` to see the results reported in this issue.


>>> second = shlex.shlex('word1 word2,\nword3', punctuation_chars=',')
>>> second.get_token()
'word1'
>>> second.lineno
1
>>> second.get_token()
'word2'
>>> second.lineno
1
>>> second.get_token()
','
>>> second.lineno
2
>>>


Closing this as a duplicate of #1521950.