Hi, I'm working on a class which implements the __contains__ method but the way I would like it to work is by generating an object that will be evaluated later.

It'll return a custom object instead of True/False


class C:
    def __contains__(self, x):
        return "I will evaluate this thing later... Don't bother now"


but when I do:


>>> 1 in C()
True


It seems to evaluate the answer with bool!

Reading the docs (http://docs.python.org/py3k/reference/expressions.html#membership-test-details) It says:

"`x in y` is true if and only if `y.__contains__(x)` is true."

It looks like the docs doesn't match the code and the code is trying to mimic the behavior of lists/tuples where "x in y" is the same as

any(x is e or x == e for e in y)

and always yield True or False.

There is a reason why it is that way?


Thanks!

"an object is true" is a short way of saying "bool(obj) is True".  So the docs match the behavior.

Returning the actual object instead of True/False from the "in" operator is a feature request.

@Georg Brandl
Oh sorry, now I see... true != True

But still, why is that the default behavior? Shouldn't it use whatever the method returns?

Well, usually what you want *is* a boolean indicating whether the element is in the collection or not.

Being able to overload "in", mostly for metaprogramming purposes, is a request that probably nobody thought of when implementing "in".

I think the idea has some merit. I think it should be well vetted on python-ideas, though. One thing that will certianly weigh against it is that implementation would not be trivial.

I see that every other comparison operator (<, >, <=, >=, ==, !=) except for `is` work the way I expect and is able to return anything.

e.g.

>>> numpy.arange(5) < 3
array([ True,  True,  True, False, False], dtype=bool)

I didn't checked the code (and probably I'm talking nonsense), but seems like the `in` operator has an extra call to `PyObject_IsTrue` that maybe could be dropped?

Of course it can break code relying on `x in y` being True/False but it would only happen on customized classes.

Another option that won't break code is to add a different method to handle these cases. Something like "__contains_non_bool__", but that'd be a big api change.

2011/12/28 João Bernardo <report@bugs.python.org>:
>
> João Bernardo <jbvsmo@gmail.com> added the comment:
>
> I see that every other comparison operator (<, >, <=, >=, ==, !=) except for `is` work the way I expect and is able to return anything.
>
> e.g.
>
>>>> numpy.arange(5) < 3
> array([ True,  True,  True, False, False], dtype=bool)
>
> I didn't checked the code (and probably I'm talking nonsense), but seems like the `in` operator has an extra call to `PyObject_IsTrue` that maybe could be dropped?

I'm not sure what you're referring to, but I doubt that would do the job.

>
> Of course it can break code relying on `x in y` being True/False but it would only happen on customized classes.
>
> Another option that won't break code is to add a different method to handle these cases. Something like "__contains_non_bool__", but that'd be a big api change.

And completely hideous.

Using my poor grep abilities I found that on Objects/typeobject.c
(I replaced some declarations/error checking from the code with ...)

static int
slot_sq_contains(PyObject *self, PyObject *value) {
    ...
    func = lookup_maybe(self, "__contains__", &contains_str);
    if (func != NULL) {
        ...
        res = PyObject_Call(func, args, NULL);
        ...
        if (res != NULL) {
            result = PyObject_IsTrue(res);
            Py_DECREF(res);
        }
    }
    else if (! PyErr_Occurred()) {
        /* Possible results: -1 and 1 */
        result = (int)_PySequence_IterSearch(self, value,
                                         PY_ITERSEARCH_CONTAINS);
    }
}

    
I don't know if I'm in the right place, but the function returns `int` and evaluates the result to 1 or 0 if __contains__ is found.

I also don't know what SQSLOT means, but unlike the other operators (which are defined as TPSLOT), `slot_sq_contains` is a function returning "int" while `slot_tp_richcompare` returns "PyObject *".

Why is that defined that way?

It's defined that way because it's a slot returning a bool, so it doesn't need to return anything except for 0 or 1.

Changing this to return a PyObject would mean that every extension module (i.e. module written in C) that defines a custom __contains__ would need to be adapted.  That is the non-trivial implementation that Benjamin was talking about.

-1 on this proposal.  It has everyone paying a price for a questionable feature that would benefit very few.

For what it's worth I proposed this on -ideas a while ago, the sticking points were what does `not in` do (no one had an answer anyone was happy with for this), and do we need a way to override it from the other perspective (e.g. if I want to do `SpecialObj() in [1, 2, 3]`, is there a way to do that?).

The problem with `not in` is because it must evaluate the result. It's not just another operator like "==" and "!=".

Looks like we're suffering from premature optimization and now it would break a lot of code to make it good.

For my application, I created a different method to generate the object (Not as good as I wanted, but there's no option right now):

`MyClass().has(1)` instead of `1 in MyClass()`

So, if no one comes up with a better idea, this issue should be closed.

Thanks