See the example below (also attached).

First example: The lineno reported just after "word2" is pulled is 2.
Second example: The lineno reported just after "," is pulled is still 1.

This behaviour seems inconsistent. The lineno should increment either when the last token of a line is pulled, or after the first token from the next line (in my opinion preferably the former). It should not have different bahaviour depending on what type of token that is (alpha vs. comma).

I have repeated this on 

Also, does Issue 16121 relate to this?


#!/usr/bin/env python
import shlex

first = shlex.shlex("word1 word2\nword3")
print (first.get_token())
print (first.get_token())
print ("line no", first.lineno)
print ("")

second = shlex.shlex("word1 word2,\nword3")
print (second.get_token())
print (second.get_token())
print (second.get_token())
print ("line no", second.lineno)


# Output:
# word1
# word2
# line no 2
#
# word1
# word2
# ,
# line no 1