Does pytest have an assertItemsEqual / assertCountEqual equivalent

Question

unittest.TestCase has an assertCountEqual method (assertItemsEqual in Python 2, which is arguably a better name), which compares two iterables and checks that they contain the same number of the same objects, without regard for their order.

Does pytest provide something similar? All of the obvious alternatives (e.g. calling set(x), sorted(x) or Counter(list(x)) on each side as mentioned in the documentation) don't work, because the things I'm comparing are lists of dictionaries, and dictionaries aren't hashable.

Why doesn't `sorted` work? It doesn't require anything to be hashable. — The Compiler, Jan 12 '17 at 19:33
I agree with @TheCompiler. The only thing you need is specifying `key` parameter to compare dictionaries: http://stackoverflow.com/questions/72899/how-do-i-sort-a-list-of-dictionaries-by-values-of-the-dictionary-in-python. — Piotr Dawidiuk, Jan 12 '17 at 19:35
@TheCompiler No, you're right, but it does require the things you're sorting to be comparable with >, which dicts aren't. — Daisy Leigh Brenecki, Jan 12 '17 at 19:36
Related pytest issue linking to this very question, closed as "probably never will do" but might be a future source of workarounds (community plugins?) https://github.com/pytest-dev/pytest/issues/5548 — N1ngu, Feb 03 '22 at 11:37

theY4Kman · Answer 1 · 2020-06-18T08:00:17.340

pytest does not provide an assertCountEqual, but we can just use unittest's:

import unittest

def test_stuff():
    case = unittest.TestCase()
    a = [{'a': 1}, {'b': 2}]
    b = [{'b': 2}]
    case.assertCountEqual(a, b)

And the output is decent, too

$ py.test
============================= test session starts ==============================
platform linux -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /home/they4kman/.virtualenvs/tmp-6626234b42fb350/src, inifile:
collected 1 item

test_stuff.py F

=================================== FAILURES ===================================
__________________________________ test_stuff __________________________________

    def test_stuff():
        case = unittest.TestCase()
        a = [{'a': 1}, {'b': 2}]
        b = [{'b': 2}]
>       case.assertCountEqual(a, b)

test_stuff.py:7:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.6/unittest/case.py:1182: in assertCountEqual
    self.fail(msg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <unittest.case.TestCase testMethod=runTest>
msg = "Element counts were not equal:\nFirst has 1, Second has 0:  {'a': 1}"

    def fail(self, msg=None):
        """Fail immediately, with the given message."""
>       raise self.failureException(msg)
E       AssertionError: Element counts were not equal:
E       First has 1, Second has 0:  {'a': 1}

/usr/lib/python3.6/unittest/case.py:670: AssertionError
=========================== 1 failed in 0.07 seconds ==========================

Side note: the implementation of assertCountEqual includes a branch specifically for unhashable types, which does a bunch of bookkeeping and compares each item with every other item.

This technique is the way to go if you need to use any rich assertions from `unittest.TestCase` or any of its existing arbitrary framework subclasses. Most can be replaced by bare assert instructions in pytest, but some can't and this answer is very powerful. — N1ngu, Feb 03 '22 at 11:55

N1ngu · Answer 2 · 2022-02-03T20:32:37.477

Alternatively, unittests.util.unorderable_list_difference can be used for a more pytesthonic formula. It returns two lists containing the missing and the unexpected items.

from unittests.util import unorderable_list_difference


def test_lists_count_equal():
   (missing, unexpected) = unorderable_list_difference(
       [1, 2, 3, 4],
       [1, 2, 3, 3]
   )
   assert (missing, unexpected) == ([], [])

yielding

>       assert (missing, unexpected) == ([], [])
E       assert ([4], [3]) == ([], [])
E         At index 0 diff: [4] != []
E         Full diff:
E         - ([], [])
E         + ([4], [3])
E         ?   +    +

You can also opt in for unittests.util.sorted_list_difference if your datasets are already sorted.

Caveats

I can't find this methods among the python library doc, so I can't tell about their stability guarantees.
unorderable_list_difference docstring states it has O(n*n) performance so be careful with big datasets.

jpsecher · Answer 3 · 2022-01-04T12:08:24.463

You can make a generic helper function that works for most cases:

def items_equal(xs, ys):
    if isinstance(xs, dict) and isinstance(ys, dict):
        if len(xs) != len(ys):
            return False
        for key in xs.keys():
            try:
                if not items_equal(xs[key], ys[key]):
                    return False
            except KeyError:
                return False
        return True
    elif isinstance(xs, list) and isinstance(ys, list):
        if len(xs) != len(ys):
            return False
        sxs = xs
        sys = ys
        try:
            sxs = sorted(xs)
            sys = sorted(ys)
            for x, y in zip(sxs, sys):
                if not items_equal(x, y):
                    return False
        except TypeError:
            ys_copy = ys.copy()
            for x in xs:
                matches = [i for i, y in enumerate(ys_copy) if items_equal(x, y)]
                if len(matches):
                    del ys_copy[matches[0]]
                    continue
                else:
                    return False
        return True
    else:
        return xs == ys

Here are the tests:

def test_items_equal_simple():
    assert items_equal(1, 1)
    assert not items_equal(1, 2)
    assert items_equal('Hello', 'Hello')
    assert not items_equal('', 'no')


def test_items_equal_complex():
    assert items_equal([1, 2, 3], [3, 2, 1])
    assert not items_equal([1, 2, 3], [3, 2, 1, 4])
    assert not items_equal(
        {'one': 1},
        {'two': 2}
    )
    assert items_equal(
        {'one': 1, 'two': 2},
        {'two': 2, 'one': 1}
    )
    assert items_equal(
        {'one': [1, 2, 3], 'two': [3]},
        {'two': [3], 'one': [3, 1, 2]},
    )
    assert items_equal(
        {'four': {'one': 1, 'two': 2}, 'five': 5},
        {'five': 5, 'four': {'two': 2, 'one': 1}}
    )
    assert not items_equal(
        {'four': {'one': 1, 'two': 2, 'eight': 8}, 'five': 5},
        {'five': 5, 'four': {'two': 2, 'one': 1}}
    )


def test_items_equal_dict_in_list():
    assert items_equal(
        [{'one': 1, 'two': 2}, {'five': 5}],
        [{'one': 1, 'two': 2}, {'five': 5}]
    )
    assert items_equal(
        [{'one': 1, 'two': 2}, {'five': 5}],
        [{'five': 5}, {'two': 2, 'one': 1}]
    )


def test_items_equal_dict_list_dict_list():
    assert items_equal(
        {'a': [{'d': [{'h': [1, 2, 3], 'i': [4, 5, 6]}], 'e': [{'j': [7, 8], 'k': [9, 10]}]}], 'b': [{'f': [11, 12], 'g': [13, 14]}]},
        {'b': [{'f': [12, 11], 'g': [14, 13]}], 'a': [{'d': [{'i': [6, 5, 4], 'h': [3, 2, 1]}], 'e': [{'k': [10, 9], 'j': [8, 7]}]}]},
    )
    assert not items_equal(
        {'a': [{'d': [{'h': [1, 2]}]}]},
        {'a': [{'d': [{'h': [1, 2, 3]}]}]}
    )
    assert not items_equal(
        [{'one': 1}, {'two': 2}],
        [{'one': 1}, {'one': 1}]
    )
    assert not items_equal(
        [{'one': 1}, {'one': 1}],
        [{'one': 1}, {'two': 2}]
    )

Does pytest have an assertItemsEqual / assertCountEqual equivalent

3 Answers3

Caveats

Linked