The current behavior of "assertSameElements" is not correctly documented.

The unit test confirm that it is a tested behaviour.

However it seems more useful to provide a method which compares the actual count of each element in both sequences.
There's already some use cases in the stdlib test suite.

Proposed behavior:

Success: assertSameElements([1, None, None], [None, 1, None])
Failure: assertSameElements([1, None, None], [1, 1, 1, None])

Patch which adds method "assertItemsEqual".

However we could reuse the "assertSameElements" name, and replace the current implementation: it is still the alpha stage.

I like that this patch adds a new method with the different behavior.  They are both distinct and desirable behaviors.  The existing behavior has already been shipped in Python 3.1 so it should not change.  Also it is already in use via the http://code.google.com/p/python-unittest-backport/ project.

Also, I would put the assertItemEquals unit tests in their own method instead of at the end of the existing one.

Just out of curiosity, what are the use cases for assertSameElements? If one has two objects, doesn't care about the order and the duplicates then these two objects are most likely sets and assertSetEqual can be used instead.
OTOH checking if two sequences have the same elements regardless of their order seems more common, and I don't see any method that does it (I was expecting assertSameElements to do that).

I would be -0 to add another method -- the API is already too confusing IMHO (is not so easy to remember if it's assertSameList, assertSameLists, assertListEqual, assertListsEqual; also note the 'assertItemsEqual' in msg98697, 'assertItemEquals' in msg98717).
Maybe a check_order arg in assertSequenceEqual (or was it assertSameSequence? ;) would be better, or at least more explicit.

Damn. If assertSameElements has already shipped with the existing implementation then we can't change it. The documentation really needs clarifying to make it clear that it ignores duplicates.

I would be happy with a 'check_order' argument to assertSequenceEqual defaulting to True, and slightly prefer that to a new method.

FWIW the http://code.google.com/p/python-unittest-backport/ project is quite out of date. I am intending to do my own port (before PyCon) that subclasses unittest so that it can interoperate with code using the Python 2.4-2.6 standard library unittest. I may get in touch with the project owner to see if I can join him on his project.

fwiw - The documentation was updated in trunk, py3k and release31-maint to mention this behavior of assertSameElements.

Assigning to Michael for a decision on whether or not to add to the API.

python-unittest-backport has since been supplanted by Michael's http://pypi.python.org/pypi/unittest2/.

Patch with documentation, tests, Misc/NEWS, following a discussion with Michael, Ezio and JP.

To summarize, the possible behaviors are:
1) check  order, check  duplicates (i.e. assertSequenceEqual);
2) check  order, ignore duplicates (probably useless);
3) ignore order, check  duplicates (useful but missing);
4) ignore order, ignore duplicates (i.e. assertSameElements now);

The possible solutions are:
a) change assertSameElements to match behavior 3 (see Florent patch):
   pros: implements 3 (i.e. the expected behavior); 4 can be replaced easily *; shorter function call for common use;
   cons: breaks compatibility; the name is still kind of confusing;
b) add check_order to assertSequenceEqual, leave assertSameElements unchanged:
   pros: covers 1, 3 and 4; backward compatible;
   cons: assertSameElements is still confusing;
c) add check_order to assertSequenceEqual, deprecate and then remove assertSameElements:
   pros: covers 1 and 3; removes a confusing function; 4 can be replaced easily *;
   cons: deprecates a quite new function;
d) add check_duplicates to assertSameElements:
   pros: covers 1, 3 and 4; backward compatible (with default == False);
   cons: assertSameElements would be even more confusing;

* assertSameElements can be replaced by assert[Set]Equal(set(a), set(b)), but this doesn't work with unhashable elements (they worked with assertSameElements).

I like c) because it removes a confusing function, and simplifies the API of unittest that IMHO it's already growing too complex (even if it would be handy to have a function that does 3 directly without having to write check_order=False).

Looking through the thousands of uses of assertSameElements in our internal code base at work I see many uses of it that are likely not hashable items in the sequences being compared.

The largest use of course is with lists and tuples of hashables where another assert method may have even made more sense, but that is not the only use.

As documented in Python 3.1 I think the behavior of assertSameElements is accurate and makes sense.  We should add an extra note to the documentation to explicitly mention that it does not care how many of a given element occur in either sequence.  [0, 1, 1] and [0, 0, 1] do in fact have the same elements.

If you want a different behavior please add that as a feature.  As such, Ezio's option (b) and (d) are the only ones I'm in favor of.

Gregory P. Smith wrote:
> Looking through the thousands of uses of assertSameElements in our internal
> code base at work I see many uses of it that are likely not hashable items
> in the sequences being compared.

The method assertSameElements will still support unhashable elements.
In this case the fixed behaviour will be harmless for your code base,
like for many other users. Example with unhashable entries:

    assertSameElements(list([3], [None], [3]), tuple([None], [3], [3]))
    ==> pass

    assertSameElements(list([3], [None], [3]), tuple([None], [None], [3]))
    ==> fails

The patch only fix the 2nd case here. Before the patch it didn't fail on
this case.


I still don't see real cases where we need that [0, 0, 1] and [0, 1, 1]
compare equal.

And this enhancement will be less surprising for the user.

Currently, if someone uses "assertSameElements" it could expect that it
fails when the count of elements is not the same. And it will never notice
that he's wrong and that his unittest have a flaw.
In a sense, it is not helpful for the end user to preserve the current
behaviour. There's a risk that tests will not fail when they should.

Some real use case for the fixed behavior:
 - the patch for #7092 has a workaround to fix the comparison locally,
   both in test_decimal and test_set
 - test_cgi was changed, before noticing that SameElements has a flaw:
http://svn.python.org/view/python/trunk/Lib/test/test_cgi.py?r1=77871&r2=77870


But there's many snippets in the test suite which look like:

  expected.sort()
  actual.sort()
  self.assertEqual(expected, actual)

Since sorting heterogeneous sequence is no longer supported in 3.x, it is painful to workaround this in each test case.

Committed in revision 79132.

For Python 2.7 I renamed assertSameElements to assertItemsEqual (assertSameElements has never been released on 2.X). In 3.2 assertItemsEqual will be new and assertSameElements will be de-documented / deprecated.

Committed to Py3k in revision 79141, revision 79142 and  revision 79143.