Python Counter with defaultdict(int) behaviour

Question

Consider the following piece of code:

from collections import Counter
from cytoolz import merge_with

my_list = ["a", "b", "a", "a", "c", "d", "b"]
my_dict = {"a" : "blue", "b" : "green", "c" : "yellow", "d" : "red", "e" : "black"}

pair_dict = merge_with(tuple, my_dict, Counter(my_list))

I obtain the following pair_dict:

{'a': ('blue', 3),
 'b': ('green', 2),
 'c': ('yellow', 1),
 'd': ('red', 1),
 'e': ('black',)}

In my real case application I need the values in my pair_dict to be pairs, so pair_dict["e"] should be ('black', 0).

It would be very convenient if I could have a class that extends Counter with the nice behaviour of a defaultdict(int).

Is this easily done?

I naïvely tried the following:

class DefaultCounter(defaultdict, Counter):
    pass

pair_dict = merge_with(tuple, my_dict, DefaultCounter(my_list))

But I get TypeError: first argument must be callable or None. I guess this is due to the fact that defaultdict expects a factory function.

So I tried the following:

pair_dict = merge_with(tuple, my_dict, DefaultCounter(int, my_list))

This results in ValueError: dictionary update sequence element #0 has length 1; 2 is required.

I also tried class DefaultCounter(Counter, defaultdict) but this does not have the desired effect: pair_dict["e"] is still ('black',).

Probably something else should be done in the definition of the class.

So I tried to adapt this answer:

class DefaultCounter(Counter):
    def __missing__(self, key):
        self[key] = 0
        return 0

But this also doesn't have the desired effect (pair_dict["e"] still misses a second element).

Edit: `Counter` already behaves as `defaultdict(int)`, but `merge_with` does not trigger this behaviour.

As suggested in the comments, a Counter already has the desired behaviour:

my_counts = Counter(my_list)
assert my_counts["e"] == 0

The issue may actually lie in the way merge_with works: It doesn't trigger the desired defaultdict behaviour.

This is verified by the following test using a defaultdict instead of a Counter:

from collections import defaultdict
my_counts = defaultdict(int)
for letter in my_list:
    my_counts[letter] += 1
pair_dict = merge_with(tuple, my_dict, my_counts)
assert pair_dict["e"] == ('black',)

One must therefore ensure that all keys have been created in the Counter before merging with the other dict, for instance using this trick.

`Counter` already exhibits that behaviour. Try `Counter('')['a']`. I suspect the problem is that `merge_with` is checking something like `key in d` and so is never trying `Counter(myList)['e']` to get that zero value. Try to write a different function to replace `tuple` that will fill in mizzing values. Or you might be able to write a `Counter` subclass that overwrite `__contains__` to return `True` for everything — Patrick Haugh, Nov 20 '17 at 16:33

score 1 · Answer 1 · answered Nov 20 '17 at 16:25

1

Not what you asked for, but 1 option would be to initialise a Counter with the dict keys and then update it with the list and finally use a dict comprehension to get your desired output:

>>> c = Counter(my_dict.keys())
>>> c.update(my_list)
>>> {k:(my_dict[k],v-1) for k,v in c.items()}
{'a': ('blue', 3), 'b': ('green', 2), 'c': ('yellow', 1), 'd': ('red', 1), 'e': ('black', 0)}

answered Nov 20 '17 at 16:25

Chris_Rands

38,994
14
83
119

This works, and I upvoted despite not being exactly what I asked for. Strangely, someone downvoted without leaving any comments. What is the issue with this approach? Currently, none of the other answers is exactly what I asked for either, and they did not get downvoted. – bli Nov 20 '17 at 16:47
Thanks, I'm not sure why it was downvoted, an alternative would be `c = Counter(); for k in my_dict: c[k] = 0` and then update without the need for subtracting 1 – Chris_Rands Nov 20 '17 at 18:39

score 1 · Answer 2 · answered Nov 27 '17 at 13:40

Not a direct answer, but some other ways to approach this porblem. join:

from toolz import first, join

{k: (v, c) for (_, c), (k, v) in join(
    leftkey=first, leftseq=Counter(my_list).items(), 
    rightkey=first, rightseq=my_dict.items(),
    left_default=(None, 0))}

and merge_with:

from toolz import *

merge_with(
    tuple, 
    my_dict, 
    reduce(
        lambda acc, x: update_in(acc, x, identity, 0),
        my_dict.keys(), 
        Counter(my_list)))

score 0 · Answer 3 · answered Nov 20 '17 at 16:25

A dictionary comprehension should do similar to list comprehension.

my_list = ["a", "b", "a", "a", "c", "d", "b"]
my_dict = {"a" : "blue", "b" : "green",
           "c" : "yellow", "d" : "red", "e" : "black"}

res={key:(my_dict[k], my_list.count(k)) for k in my_dict}
res

# {'a': ('blue', 3), 'b': ('green', 2), 'c': ('yellow', 1),
#  'd': ('red', 1), 'e': ('black', 0)}

score 0 · Accepted Answer · answered Nov 20 '17 at 18:17

Combining this answer with the use of merge_with, I came up with the following solution:

from collections import Counter
from cytoolz import merge_with

my_list = ["a", "b", "a", "a", "c", "d", "b"]
my_dict = {
    "a" : "blue", "b" : "green", "c" : "yellow", "d" : "red", "e" : "black"}
my_counts = Counter(my_dict.keys()).update(my_list)
pair_dict = merge_with(
    tuple, my_dict,
    {k : v - 1 for (k, v) in my_counter.items()})
assert pair_dict["e"] == ('black', 0)

Python Counter with defaultdict(int) behaviour

Edit: Counter already behaves as defaultdict(int), but merge_with does not trigger this behaviour.

4 Answers4

Edit: `Counter` already behaves as `defaultdict(int)`, but `merge_with` does not trigger this behaviour.