classification
Title: Reconsider comparison chaining for containment tests
Type: behavior Stage: patch review
Components: Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, mark.dickinson, ncoghlan, serhiy.storchaka, taleinat
Priority: low Keywords: patch

Created on 2017-11-17 07:43 by ncoghlan, last changed 2018-09-12 12:10 by mark.dickinson.

Pull Requests
URL Status Linked Edit
PR 4501 open serhiy.storchaka, 2017-11-22 15:44
Messages (12)
msg306411 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-11-17 07:43
(I thought there was an open low priority issue for this, but I can't find it, so filing a new one)

Currently, "in" and "not in" are classified as comparison operations in the language grammar, even though they're not actually documented as such (see https://bugs.python.org/issue28617 in relation to the latter point).

Issue reports like https://bugs.python.org/issue30965, user questions like https://stackoverflow.com/questions/45180899/unexpected-result-from-in-operator-python/45180967, and behaviour puzzles like https://github.com/cosmologicon/pywat#operator-precedence suggest that the existing behaviour isn't particular intuitive to users.

At the language design level (as far as I am aware), the benefit of treating "in" and "not in" as comparison operators is to ensure they share a precedence level with the other comparisons.

While this is mostly a pretty harmless quirk, I think it's weird enough and useless enough for us to at least consider refining the Grammar such that "in" and "not in" live at the same level as other comparison operators, but *don't* participate in comparison chaining (i.e. "a == b in c" and "a in c == b" would both be syntax errors that required parentheses to disambiguate the desired associativity).
msg306413 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-11-17 07:47
https://bugs.python.org/issue14247 is another older issue related to this point (again related to a user finding the current behaviour genuinely unintuitive)
msg306415 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-11-17 09:12
`a in b < c` makes sense if b and c are sets.
msg306418 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-11-17 09:30
Aye, there are definitely cases where the answer isn't nonsense.

Even for sets though, "a in b < c" isn't an intuitive spelling of "a in b and b < c" the way that "a < b < c" is an intuitive spelling of an ordering relation.

Hence filing this as a low priority issue - it's a weird quirk, and potentially worth changing to avoid a particular "Wat?" moment when folks first see it, but it isn't a bug magnet the way some other historical constructs have been.
msg306419 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-11-17 09:33
Just a note on why I find "a in b < c" unintuitive, even for sets: my brain initially wanted to read it as equivalent to "a in b & c".
msg306734 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-11-22 15:47
PR 4501 makes the parser emitting a SyntaxWarning for chained `in` and `not in` comparison operators. In future it can be turned into a SyntaxError.

But this should be first discussed on Python-Dev.
msg306760 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-11-22 23:18
If we do decide to do this, I'm inclined to eventually make the change at the Grammar level rather than the AST level.

Current:

    comparison: expr (comp_op expr)*
    comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'

Future:

    comparison: expr (comp_op expr)* | (containment_check)
    comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'is'|'is' 'not'
    containment_check: ('in'|'not' 'in') expr

However, we'd still need an intermediate step like your PR in order to emit SyntaxWarning while still retaining the current semantics.
msg306848 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-11-23 17:34
I'm reading the list of most rated in a week questions on StackOverflow (https://python-weekly.blogspot.com/), and it seems to me that the question about chained "in" and "not in" is raised every few months. This may be the one of the most confusing quirks.
msg316318 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-05-09 09:10
On other hand, beginners are confused even by chained "==". There are regular reports about this on the tracker and questions on Python-related resources.

I have found the chained "in" is useful in the following case

    len(string) >= 2 and string[0] == string[-1] in ("'", '"')

for testing if the string is quoted with a single or double quotes.
msg324687 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2018-09-06 12:18
Has this been discussed on python-dev? If so, what was the result?
msg324789 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-07 20:02
This had not been discussed.
msg325076 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2018-09-11 22:09
I agree that it would be less confusing if `in`/`not in` did not allow chaining, the occasional (surely very rare) useful example notwithstanding.

Then again if we're going to forbid (or even discourage) unusual combinations we might also want to frown at `a < b > c` -- surely in mathematical circles the chaining always goes in one direction only, e.g. `a < b <= c` or `a >= b ==c > d`.

Finally as long as we're refining the terminology, maybe we could strive to distinguish "comparison" (`==` and `!=`) from "ordering" (`<`, `<=`, `>`, `>=`)?
History
Date User Action Args
2018-09-12 12:10:47mark.dickinsonsetnosy: + mark.dickinson
2018-09-11 22:09:48gvanrossumsetnosy: + gvanrossum
messages: + msg325076
2018-09-07 20:02:24serhiy.storchakasetmessages: + msg324789
2018-09-06 12:18:56taleinatsetnosy: + taleinat
messages: + msg324687
2018-05-09 09:10:37serhiy.storchakasetmessages: + msg316318
2017-11-23 17:34:03serhiy.storchakasetmessages: + msg306848
2017-11-22 23:18:19ncoghlansetmessages: + msg306760
2017-11-22 15:47:58serhiy.storchakasetmessages: + msg306734
2017-11-22 15:44:08serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request4439
2017-11-17 09:33:55ncoghlansetmessages: + msg306419
2017-11-17 09:30:58ncoghlansetmessages: + msg306418
2017-11-17 09:12:19serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg306415
2017-11-17 07:47:57ncoghlansetmessages: + msg306413
2017-11-17 07:43:17ncoghlancreate