I think I'm suffering the same problem in some small programs that use shlex:


>>> import shlex

>>> text = "python and shlex"
>>> shlex.split(text)
['python', 'and', 'shlex']

>>> text = u"python and shlex"
>>> shlex.split(text)
['p\x00\x00\x00y\x00\x00\x00t\x00\x00\x00h\x00\x00\x00o\x00\x00\x00n\x00\x00\x00', '\x00\x00\x00a\x00\x00\x00n\x00\x00\x00d\x00\x00\x00', '\x00\x00\x00s\x00\x00\x00h\x00\x00\x00l\x00\x00\x00e\x00\x00\x00x\x00\x00\x00']


 I'm currently using the following "basic" workaround (while assuming that my strings have only ascii chars):

>>> [ x.replace("\0", "") for x in shlex.split(text) ]
['python', 'and', 'shlex']

 It would be very nice if shlex could work with unicode strings ...

 Thanks.