Attached a program which shows the relevant behaviour:

import shlex

tests = [ "foo#bar", "foo #bar" ]

for t in tests:
  print "%s -> %s" % (t,[x for x in shlex.shlex(t,posix=True)])

results in

$ python lexer_test.py
foo#bar -> ['foo']
foo #bar -> ['foo']

(expected of course is ['foo#bar'] on the first line).