Logged In: YES 
user_id=593130

To me, this is not a bug report but at best an RFE.  The 
reported behavior is what I would expect.  I read both 
module doc and the referenced web page and further links.  
The doc does not mention Unicode as allowed and the 300 
registered UserAgents at
http://www.robotstxt.org/wc/active/html/index.html
all have ascii names.

So I recomment closing this as a bug report but will give 
ML a chance to respond.  If switched instead to Feature 
Request, I would think it would need some 'in the wild' 
evidence of need.