This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: assertSameElements([0, 1, 1], [0, 0, 1]) does not fail
Type: enhancement Stage: resolved
Components: Versions: Python 3.2, Python 2.7
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: michael.foord Nosy List: eric.araujo, ezio.melotti, flox, gregory.p.smith, michael.foord
Priority: normal Keywords: patch

Created on 2010-02-01 21:43 by flox, last changed 2022-04-11 14:56 by admin. This issue is now closed.

File name Uploaded Description Edit
issue7832_assertSameElements_v2.diff flox, 2010-03-08 23:14 Patch, apply to 2.x
Messages (13)
msg98692 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-02-01 21:43
The current behavior of "assertSameElements" is not correctly documented.

The unit test confirm that it is a tested behaviour.

However it seems more useful to provide a method which compares the actual count of each element in both sequences.
There's already some use cases in the stdlib test suite.

Proposed behavior:

Success: assertSameElements([1, None, None], [None, 1, None])
Failure: assertSameElements([1, None, None], [1, 1, 1, None])
msg98697 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-02-01 22:54
Patch which adds method "assertItemsEqual".

However we could reuse the "assertSameElements" name, and replace the current implementation: it is still the alpha stage.
msg98717 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2010-02-02 05:06
I like that this patch adds a new method with the different behavior.  They are both distinct and desirable behaviors.  The existing behavior has already been shipped in Python 3.1 so it should not change.  Also it is already in use via the project.

Also, I would put the assertItemEquals unit tests in their own method instead of at the end of the existing one.
msg98722 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2010-02-02 06:35
Just out of curiosity, what are the use cases for assertSameElements? If one has two objects, doesn't care about the order and the duplicates then these two objects are most likely sets and assertSetEqual can be used instead.
OTOH checking if two sequences have the same elements regardless of their order seems more common, and I don't see any method that does it (I was expecting assertSameElements to do that).

I would be -0 to add another method -- the API is already too confusing IMHO (is not so easy to remember if it's assertSameList, assertSameLists, assertListEqual, assertListsEqual; also note the 'assertItemsEqual' in msg98697, 'assertItemEquals' in msg98717).
Maybe a check_order arg in assertSequenceEqual (or was it assertSameSequence? ;) would be better, or at least more explicit.
msg98728 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-02-02 12:55
Damn. If assertSameElements has already shipped with the existing implementation then we can't change it. The documentation really needs clarifying to make it clear that it ignores duplicates.

I would be happy with a 'check_order' argument to assertSequenceEqual defaulting to True, and slightly prefer that to a new method.

FWIW the project is quite out of date. I am intending to do my own port (before PyCon) that subclasses unittest so that it can interoperate with code using the Python 2.4-2.6 standard library unittest. I may get in touch with the project owner to see if I can join him on his project.
msg100224 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2010-02-28 22:04
fwiw - The documentation was updated in trunk, py3k and release31-maint to mention this behavior of assertSameElements.

Assigning to Michael for a decision on whether or not to add to the API.

python-unittest-backport has since been supplanted by Michael's
msg100677 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-03-08 23:14
Patch with documentation, tests, Misc/NEWS, following a discussion with Michael, Ezio and JP.
msg100685 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2010-03-09 00:35
To summarize, the possible behaviors are:
1) check  order, check  duplicates (i.e. assertSequenceEqual);
2) check  order, ignore duplicates (probably useless);
3) ignore order, check  duplicates (useful but missing);
4) ignore order, ignore duplicates (i.e. assertSameElements now);

The possible solutions are:
a) change assertSameElements to match behavior 3 (see Florent patch):
   pros: implements 3 (i.e. the expected behavior); 4 can be replaced easily *; shorter function call for common use;
   cons: breaks compatibility; the name is still kind of confusing;
b) add check_order to assertSequenceEqual, leave assertSameElements unchanged:
   pros: covers 1, 3 and 4; backward compatible;
   cons: assertSameElements is still confusing;
c) add check_order to assertSequenceEqual, deprecate and then remove assertSameElements:
   pros: covers 1 and 3; removes a confusing function; 4 can be replaced easily *;
   cons: deprecates a quite new function;
d) add check_duplicates to assertSameElements:
   pros: covers 1, 3 and 4; backward compatible (with default == False);
   cons: assertSameElements would be even more confusing;

* assertSameElements can be replaced by assert[Set]Equal(set(a), set(b)), but this doesn't work with unhashable elements (they worked with assertSameElements).

I like c) because it removes a confusing function, and simplifies the API of unittest that IMHO it's already growing too complex (even if it would be handy to have a function that does 3 directly without having to write check_order=False).
msg100692 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2010-03-09 01:46
Looking through the thousands of uses of assertSameElements in our internal code base at work I see many uses of it that are likely not hashable items in the sequences being compared.

The largest use of course is with lists and tuples of hashables where another assert method may have even made more sense, but that is not the only use.

As documented in Python 3.1 I think the behavior of assertSameElements is accurate and makes sense.  We should add an extra note to the documentation to explicitly mention that it does not care how many of a given element occur in either sequence.  [0, 1, 1] and [0, 0, 1] do in fact have the same elements.

If you want a different behavior please add that as a feature.  As such, Ezio's option (b) and (d) are the only ones I'm in favor of.
msg100700 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-03-09 06:28
Gregory P. Smith wrote:
> Looking through the thousands of uses of assertSameElements in our internal
> code base at work I see many uses of it that are likely not hashable items
> in the sequences being compared.

The method assertSameElements will still support unhashable elements.
In this case the fixed behaviour will be harmless for your code base,
like for many other users. Example with unhashable entries:

    assertSameElements(list([3], [None], [3]), tuple([None], [3], [3]))
    ==> pass

    assertSameElements(list([3], [None], [3]), tuple([None], [None], [3]))
    ==> fails

The patch only fix the 2nd case here. Before the patch it didn't fail on
this case.

I still don't see real cases where we need that [0, 0, 1] and [0, 1, 1]
compare equal.

And this enhancement will be less surprising for the user.

Currently, if someone uses "assertSameElements" it could expect that it
fails when the count of elements is not the same. And it will never notice
that he's wrong and that his unittest have a flaw.
In a sense, it is not helpful for the end user to preserve the current
behaviour. There's a risk that tests will not fail when they should.
msg100703 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-03-09 07:18
Some real use case for the fixed behavior:
 - the patch for #7092 has a workaround to fix the comparison locally,
   both in test_decimal and test_set
 - test_cgi was changed, before noticing that SameElements has a flaw:

But there's many snippets in the test suite which look like:

  self.assertEqual(expected, actual)

Since sorting heterogeneous sequence is no longer supported in 3.x, it is painful to workaround this in each test case.
msg101362 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-20 03:03
Committed in revision 79132.

For Python 2.7 I renamed assertSameElements to assertItemsEqual (assertSameElements has never been released on 2.X). In 3.2 assertItemsEqual will be new and assertSameElements will be de-documented / deprecated.
msg101386 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-20 18:10
Committed to Py3k in revision 79141, revision 79142 and  revision 79143.
Date User Action Args
2022-04-11 14:56:57adminsetgithub: 52080
2010-03-20 18:10:41michael.foordsetstatus: open -> closed
resolution: accepted
messages: + msg101386

stage: needs patch -> resolved
2010-03-20 03:03:34michael.foordsetmessages: + msg101362
2010-03-09 07:18:12floxsetmessages: + msg100703
2010-03-09 07:08:29floxlinkissue7092 dependencies
2010-03-09 06:39:02floxsetpriority: low -> normal
2010-03-09 06:28:03floxsetmessages: + msg100700
2010-03-09 01:46:44gregory.p.smithsetmessages: + msg100692
2010-03-09 00:35:28ezio.melottisetpriority: normal -> low

messages: + msg100685
2010-03-08 23:14:28floxsetpriority: low -> normal
files: + issue7832_assertSameElements_v2.diff
messages: + msg100677
2010-03-08 23:11:45floxsetfiles: - issue7832_assertItemsEqual.diff
2010-03-08 21:18:32floxlinkissue8088 superseder
2010-02-28 22:04:39gregory.p.smithsetpriority: normal -> low
assignee: gregory.p.smith -> michael.foord
messages: + msg100224
2010-02-02 12:55:29michael.foordsetmessages: + msg98728
2010-02-02 06:35:43ezio.melottisetmessages: + msg98722
stage: needs patch
2010-02-02 05:07:45gregory.p.smithsetnosy: + michael.foord
2010-02-02 05:06:29gregory.p.smithsetassignee: gregory.p.smith
messages: + msg98717
2010-02-01 22:54:43floxsetfiles: + issue7832_assertItemsEqual.diff
keywords: + patch
messages: + msg98697
2010-02-01 21:45:02eric.araujosetnosy: + eric.araujo
2010-02-01 21:43:32floxcreate