The Python 2.7.3 documentation says the following about defining __contains__:

"Called to implement membership test operators. Should return true if item is in self, false otherwise. For mapping objects, this should consider the keys of the mapping rather than the values or the key-item pairs."

This suggests that while you should return True/False, you don't need to. Also, the documentation doesn't say that if you return a value other than True or False, it will silently coerce other values to be True or False when you use "in". For example:

>>> class Foo(object):
...     def __contains__(self, item): return 42
... 
>>> foo = Foo()
>>> 3 in foo
True
>>> foo.__contains__(3)
42

When __contains__ is defined, "in" should return whatever __contains__ returns, even if this value is neither True nor False. That would be consistent with, for example, the comparison operators: You can return anything from __lt__ and "x < 4" will correctly pass along that value regardless of what it is.

The internal API is limited to flags as the sq_contains slot and the API function can only return an int:

  PyAPI_FUNC(int) PySequence_In(PyObject *o, PyObject *value)

In order to return the Python object we'd have to alter the API which we can't do as the type slot and the function is part of the stable API and ABI.

A doc update should explain why the code doesn't behave as you expect it.

http://docs.python.org/py3k/reference/datamodel.html#object.__contains__ says that __contain__ "Should return true if item is in self, false otherwise.".  Here the lowercase true and false mean any true or false value, not just True and False and it's indeed possible to return any value.

The fact that 'in' only returns True/False, possibly converting the return value of __contains__, should be documented in the 'in' documentation.
http://docs.python.org/py3k/reference/expressions.html#membership-test-details says "For user-defined classes which define the __contains__() method, x in y is true if and only if y.__contains__(x) is true.".  This could be changed to say that the return value is converted to True/False.

There was a related discussion on python-ideas a while back:

http://mail.python.org/pipermail/python-ideas/2010-July/007733.html

Thanks for passing along the thread, it was interesting. Oddly, it seemed to die off with no real resolution.

I realize it is now too late to change __contains__ to return a non-boolean value, but for reference, the reason I wanted to return something different from __contains__ was also to implement a DSL, though not a SQL-related one. Basically, I have some kind of abstract mathematical set S, and I wanted "x in S" to return a constraint that the variable x must lie in the set S for use in a larger problem description. (In fact, one could implement __contains__ so it returned True/False with a constant numeric argument and a constraint object with a variable argument.) This would have mirrored the way one would write all this mathematically.

Do you need this PEP?
http://www.python.org/dev/peps/pep-0335/

I don't, but thanks for passing that along.

Hi,

For user-defined class, it's up to the class to do the right implementation in my opinion. It's true the description is wrong though. 

x in y means that x exist inside of y (so that the execution of y.__contain__(x) is executed successfully and (I guess) doesn't return None,False or 0).

I'll modify the doc to be :
For user-defined classes which define the __contains__() method, x in y is false if y.__contains__(x) is returning either None,False or 0.  Otherwise, x in y return true.

Regards,
Eric Lafontaine

Eric: that is not precise enough, I'm afraid :)  See msg171093 for the correct documentation update.  Specifically, in returns True if __contains__ returns a true value, and False otherwise (not the difference in case, it matters).  There are more things than just None, False, and 0 that are false in Python.

Hi all,

Here are the test I've made to understand the behavior :

class Foo_42(object):
    def __contains__(self,item): return 42 
    
class Foo_neg(object):
    def __contains__(self,item): return -42 
    
class Foo_None(object):
    def __contains__(self,item): return  
    
class Foo_false(object):
    def __contains__(self,item): return False 
    
class Foo_true(object):
    def __contains__(self,item): return True

for foo in [Foo_false(),Foo_None(),Foo_neg(),Foo_true(),Foo_42()]:
    print("3 in foo:" + str(3 in foo))
    print("foo.__contains__(3)" + str(foo.__contains__(3)))
    

which output this :
3 in foo:False
foo.__contains__(3)False
3 in foo:False
foo.__contains__(3)None
3 in foo:True
foo.__contains__(3)-42
3 in foo:True
foo.__contains__(3)True
3 in foo:True
foo.__contains__(3)42

So as long as __contains__ return False or None, the 'in' operator will be False.  Otherwise true.

Hi David, sorry for the delay on my part for providing how I was getting to that conclusion.  I've also resurrected an old post as I want to start contributing more seriously :).

As this is documentation only (not changing the code behavior), I didn't take a look at the implementation of "in".

Could you enlighten me on what else would be considered "False" in this case?

Regards,
Eric Lafontaine

oh, I've got what you meant!

Proposed change :
For user-defined classes which define the __contains__() method, the in operator will convert to False "x in y" if y.__contains__(x) return False, 0 or None.  Otherwise, the in operator will return True for any other value being returned by y.__contains__(x).


Would that make more sense?
Regards,
Eric Lafontaine

>>> bool(())
False
>>> bool([])
False
>>> bool('')
False

What you want to say is that 'in' coerces the result returned by __contains__ to a boolean value.

(first time trying to reply through email)

thanks for the example and you are right :
class Foo_emptylist(object):
    def __contains__(self,item): return []

class Foo_emptydict(object):
    def __contains__(self,item): return {}

class Foo_emptystring(object):
    def __contains__(self,item): return ''

for foo in
[Foo_false(),Foo_None(),Foo_emptylist(),Foo_emptydict(),Foo_emptystring(),Foo_neg(),Foo_true(),Foo_42()]:
    print("3 in foo:" + str(3 in foo))
    print("foo.__contains__(3)" + str(foo.__contains__(3)))

3 in foo:False
foo.__contains__(3)False
3 in foo:False
foo.__contains__(3)None
3 in foo:False
foo.__contains__(3)[]
3 in foo:False
foo.__contains__(3){}
3 in foo:False
foo.__contains__(3)
3 in foo:True
foo.__contains__(3)-42
3 in foo:True
foo.__contains__(3)True
3 in foo:True
foo.__contains__(3)42

So the proposition should be this then?
For user-defined classes which define a __contains__() method, the in
operator will apply bool() on the __contains__() method.  In other words,  "x
in y" is equivalent to "bool(y.__contains__(x))" and will return False if
bool(y.__contains__(x)) is equivalent to false.

Éric Lafontaine |  Membre du Projet VUE, Groupe Contrôle
Génie électrique, 54ème promotion UdeS | Étudiant en maitrise TI à l'ETS
VAS OPS chez Bell Mobility

« Nous voulons proposer une alternative de transport en présentant un
véhicule électrique spécifiquement conçu pour les déplacements urbains. »

2017-02-07 21:10 GMT-05:00 R. David Murray <report@bugs.python.org>:

>
> R. David Murray added the comment:
>
> >>> bool(())
> False
> >>> bool([])
> False
> >>> bool('')
> False
>
> What you want to say is that 'in' coerces the result returned by
> __contains__ to a boolean value.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue16011>
> _______________________________________
>

You've got the right idea, but you are repeating yourself.  Keep it as short as possible while still conveying the correct information.  "coerce to boolean" is better than "apply bool", because the code may not in fact be using the bool function to do it.  Your "equivalent to" phrase would be OK as an alternative, but you only need to show the equivalence, no need to also explain it in words.

New changeset 0ae7c8bd614d3aa1fcaf2d71a10ff1148c80d9b5 by R. David Murray (Amit Kumar) in branch 'master':
bpo-16011 clarify that 'in' always returns a boolean value
https://github.com/python/cpython/commit/0ae7c8bd614d3aa1fcaf2d71a10ff1148c80d9b5

Thanks,

New changeset 0957f262c5e47167efd520624557aebdc61bfda8 by Mariatta in branch '3.5':
bpo-16011: clarify that 'in' always returns a boolean value (GH-152) (GH-875)
https://github.com/python/cpython/commit/0957f262c5e47167efd520624557aebdc61bfda8

New changeset c4021af50526f488c0c280e7c7eaa83ef80ae1df by Mariatta in branch '3.6':
bpo-16011: clarify that 'in' always returns a boolean value (GH-874)
https://github.com/python/cpython/commit/c4021af50526f488c0c280e7c7eaa83ef80ae1df

New changeset fd704a02ca8713b0dae644f7f182f3e3d1218dbf by Mariatta in branch '2.7':
bpo-16011: clarify that 'in' always returns a boolean value (GH-152) (GH-883)
https://github.com/python/cpython/commit/fd704a02ca8713b0dae644f7f182f3e3d1218dbf

Thanks everyone :) Patch has been merged and backported to 3.6, 3.5, and 2.7.