classification
Title: Rlcompleter.Completer does not use __dir__ magic method
Type: behavior Stage:
Components: Documentation, Extension Modules Versions: Python 3.0, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: carlj, eric.araujo, gagenellina, georg.brandl
Priority: normal Keywords: patch

Created on 2009-01-26 01:28 by carlj, last changed 2010-10-29 10:07 by admin.

Files
File name Uploaded Description Edit
rlcompleter.patch gagenellina, 2009-01-27 03:58
Messages (8)
msg80556 - (view) Author: Carl Johnson (carlj) Date: 2009-01-26 01:28
The documentation at http://docs.python.org/library/rlcompleter.html
claims that

Completer.complete(text, state)¶

   Return the state*th completion for *text.

   If called for text that doesn’t include a period character ('.'), it
will complete from names currently defined in __main__, __builtin__ and
keywords (as defined by the keyword module).

   If called for a dotted name, it will try to evaluate anything without
obvious side-effects (functions will not be evaluated, but it can
generate calls to __getattr__()) up to the last part, and find matches
for the rest via the dir() function. Any exception raised during the
evaluation of the expression is caught, silenced and None is returned.

In other words, it claims to use dir(obj) as part of the tab completion
process. This is not true (using Python 2.6.1 on OS X):

>>> class B(object):
...  def __dir__(self): return dir(u"") #Makes B objects look like strings
...
>>> b = B()
>>> dir(b)
['__add__', '__class__', '__contains__', '__delattr__', '__doc__',
'__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__',
'__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__',
'__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__',
'__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__',
'_formatter_field_name_split', '_formatter_parser', 'capitalize',
'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find',
'format', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit',
'islower', 'isnumeric', 'isspace', 'istitle', 'isupper', 'join',
'ljust', 'lower', 'lstrip', 'partition', 'replace', 'rfind', 'rindex',
'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
>>> c = rlcompleter.Completer()
>>> c.complete("b.", 0) #Notice that it does NOT return __add__
u'b.__class__('
>>> c.matches #Notice that this list is completely different from the
list given by dir(b)
[u'b.__class__(', u'b.__delattr__(', u'b.__doc__', u'b.__format__(',
u'b.__getattribute__(', u'b.__hash__(', u'b.__init__(', u'b.__new__(',
u'b.__reduce__(', u'b.__reduce_ex__(', u'b.__repr__(',
u'b.__setattr__(', u'b.__sizeof__(', u'b.__str__(',
u'b.__subclasshook__(', u'b.__class__(', u'b.__class__(',
u'b.__delattr__(', u'b.__dict__', u'b.__dir__(', u'b.__doc__',
u'b.__format__(', u'b.__getattribute__(', u'b.__hash__(',
u'b.__init__(', u'b.__module__', u'b.__new__(', u'b.__reduce__(',
u'b.__reduce_ex__(', u'b.__repr__(', u'b.__setattr__(',
u'b.__sizeof__(', u'b.__str__(', u'b.__subclasshook__(',
u'b.__weakref__', u'b.__class__(', u'b.__delattr__(', u'b.__doc__',
u'b.__format__(', u'b.__getattribute__(', u'b.__hash__(',
u'b.__init__(', u'b.__new__(', u'b.__reduce__(', u'b.__reduce_ex__(',
u'b.__repr__(', u'b.__setattr__(', u'b.__sizeof__(', u'b.__str__(',
u'b.__subclasshook__(']

Suggested course of action: 

* Change the documentation for Python 2.6/3.0.
* Update Completer to use __dir__ in Pythons 2.7/3.1 and revert the
documentation.

See
http://mail.python.org/pipermail/python-dev/2009-January/thread.html#85471
msg80615 - (view) Author: Gabriel Genellina (gagenellina) Date: 2009-01-27 03:58
This is not a bug in rlcompleter; __dir__ is returning bogus items, and 
rlcompleter checks whether there exist actually an attribute with such 
name.

Defining __getattr__ (or __getattribute__) and a matching __dir__ works 
fine:

>>> class B(object):
...   def __dir__(self):
...     return dir(object) + ["xa","xb","xc"]
...   def __getattr__(self, name):
...     if name in ["xa","xb","xc"]:
...       return None
...     raise AttributeError, name
...
>>> b = B()
>>> import rlcompleter
>>> c = rlcompleter.Completer()
>>> c.complete("b.", 0)
'b.__class__('
>>> c.matches
['b.__class__(', 'b.__delattr__(', 'b.__doc__', 'b.__format__(', 
'b.__getattribute__(', 
...
'b.xa', 'b.xb', 'b.xc', 'b.__class__(', 'b.__class__(', 
...]
>>> c.complete("b.x", 0)
'b.xa'
>>> c.matches
['b.xa', 'b.xb', 'b.xc']

Now, looking at this I saw there *is* a bug in rlcompleter, as it may 
return many duplicate items:

>>> c.complete("b.__c", 0)
'b.__class__('
>>> c.matches
['b.__class__(', 'b.__class__(', 'b.__class__(', 'b.__class__(']

The attached patch fixes that.
msg80619 - (view) Author: Carl Johnson (carlj) Date: 2009-01-27 04:39
It seems to me that it isn't tab completion's place to out think the
__dir__ method. A) Because the documentation doesn't tell you that it
does (although you are warned that it may call some stuff) and B)
because if someone set up a __dir__ method, they probably are listing
the things that they want listed for a particular reason. I think that
it would be less confusing for rlcompleter to follow the __dir__ method
when it exists and only do its own poking and prodding when it does not.
msg80623 - (view) Author: Gabriel Genellina (gagenellina) Date: 2009-01-27 06:09
This is what rlcompleter does; it uses dir() to find out what names to 
return.
Or do you mean that it should not iterate along __bases__ because this 
has already been done by dir()?
msg80625 - (view) Author: Carl Johnson (carlj) Date: 2009-01-27 06:15
I think that checking to see which things really exist with
getattr/hasattr made sense back in the days before the __dir__, since in
those days the real API for an object could diverge wildly from what was
reported by dir(object), but nowadays, if someone goes to the trouble of
defining the __dir__ method, then we should just trust that as being
"the API" and not do any other checking.
msg80626 - (view) Author: Gabriel Genellina (gagenellina) Date: 2009-01-27 06:46
The check is made to decide whether the attribute is a method or not 
(because methods get a "(" appended) -- for names that fail to exist, 
one could just omit the "(" and include the name anyway.

rlcompleter does nothing special with __dir__, it always uses dir() 
only.
msg80627 - (view) Author: Carl Johnson (carlj) Date: 2009-01-27 06:53
Ah, I see. It does a dir(obj) then tests things to see which are
callable and while it is at that, it removes the names that don't really
exist according to getattr.

Actually, can we go back to the Python 2.5 behavior? I really hate those
auto-added parentheses. For one thing, it screws it up when you do
"help(name<TAB>". Am I missing some really obvious switch that would
turn the behavior back to the old style of ignoring the
callable/non-callable thing?
msg80630 - (view) Author: Gabriel Genellina (gagenellina) Date: 2009-01-27 07:20
The current behaviour is actually a requested feature: see #449227

I see your point, it may be annoying sometimes -- but calling a method 
is far more common than just getting a reference to it, so I think the 
current behaviour is fine (I'm talking about the added "(", not the 
repeated entries, nor the unneeded __bases__ recursion, nor the deleted 
entries)
History
Date User Action Args
2010-10-29 10:07:21adminsetassignee: georg.brandl -> docs@python
2010-02-16 05:57:29eric.araujosetnosy: + eric.araujo
2009-01-27 07:20:44gagenellinasetmessages: + msg80630
2009-01-27 06:53:43carljsetmessages: + msg80627
2009-01-27 06:46:08gagenellinasetmessages: + msg80626
2009-01-27 06:15:38carljsetmessages: + msg80625
2009-01-27 06:09:03gagenellinasetmessages: + msg80623
2009-01-27 04:39:40carljsetmessages: + msg80619
2009-01-27 03:58:49gagenellinasetfiles: + rlcompleter.patch
nosy: + gagenellina
messages: + msg80615
keywords: + patch
2009-01-26 01:28:57carljcreate