342

Given a dictionary { k1: v1, k2: v2 ... } I want to get { k1: f(v1), k2: f(v2) ... } provided I pass a function f.

Is there any such built in function? Or do I have to do

dict([(k, f(v)) for (k, v) in my_dictionary.iteritems()])

Ideally I would just write

my_dictionary.map_values(f)

or

my_dictionary.mutate_values_with(f)

That is, it doesn't matter to me if the original dictionary is mutated or a copy is created.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Tarrasch
  • 10,199
  • 6
  • 41
  • 57
  • 4
    A better way of writing your example would be `dict((k, f(v)) for k, v in mydict.iteritems())`, i.e. without the square brackets, that would prevent the creation of an intermediate list via a generator. – bereal Sep 01 '12 at 15:46

8 Answers8

497

There is no such function; the easiest way to do this is to use a dict comprehension:

my_dictionary = {k: f(v) for k, v in my_dictionary.items()}

In python 2.7, use the .iteritems() method instead of .items() to save memory. The dict comprehension syntax wasn't introduced until python 2.7.

Note that there is no such method on lists either; you'd have to use a list comprehension or the map() function.

As such, you could use the map() function for processing your dict as well:

my_dictionary = dict(map(lambda kv: (kv[0], f(kv[1])), my_dictionary.iteritems()))

but that's not that readable, really.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 15
    +1: this is what I would do too. `dict(zip(a, map(f, a.values())))` is marginally shorter, but I have to think about what it's doing, and remind myself that yes, keys and values are iterated over in the same order if the dict doesn't change. I don't have to think at all about what the dictcomp is doing, and so it's the right answer. – DSM Sep 01 '12 at 15:39
  • @DSM: Yeah, the `zip(adict, map(f, adict.values())))` trick requires way too much understanding from the casual code reader, not to mention a steady hand in adding all the closing params! :-P – Martijn Pieters Sep 01 '12 at 15:40
  • `{k: f(my_dictionary[k]) for k in my_dictionary}` is a little bit shorter, but interestingly it's also a little bit slower (when timing it with `timeit`, a 500-items dict and `str()` for `f`). Don't know why. – chiborg Oct 15 '14 at 15:14
  • 2
    @chiborg: that's because rather than look up all key-value pairs in one go, you are now using number-of-keys times `my_dictionary.__getitem__` calls. – Martijn Pieters Oct 15 '14 at 15:21
  • 4
    Note that since [PEP3113](https://www.python.org/dev/peps/pep-3113/) (implemented in python 3.x) tuple parameters are not supported anymore: `lambda (k,v): (k, f(v))` is to be rewritten to something like `lambda k_v: (k_v[0], f(k_v[1]))` – normanius Jan 13 '18 at 18:53
  • @normanius: thanks for the heads-up. Yes, I'm aware that parameter unpacking is gone in Py3 but until searching on Stack Overflow becomes much, much better I don't know in what answers I used the syntax. :-) – Martijn Pieters Jan 13 '18 at 18:58
  • 2
    Why did parameter unpacking get nixed? How is that an *improvement* ? – WestCoastProjects Feb 17 '18 at 21:38
  • So I read the `pep`. At least it's admitted: it is not an *improvement* per se (on the contrary ..) - but a recognition of limitations in tooling for inferring correct types of `tuple` arguments. – WestCoastProjects Feb 17 '18 at 22:13
  • 12
    coming from an FP language, Python would seem incredibly awkward. – juanchito May 09 '18 at 20:07
  • Sometimes you've got to apply a function to both the key and the value. E.g., you want to transform the key and the value using the same generated data (faker). Each pair should have their own data generated. In this case, I see no way to use dict comprehensions, only `map`. – x-yuri Jun 18 '19 at 13:52
  • @x-yuri so combine them. Map your input iterable to faker objects and then use a dict comp to create the dict key-value pairs. – Martijn Pieters Jun 23 '19 at 18:52
  • @MartijnPieters Not that easy. My input iterable is a dict. I want to transform it to another dict (both keys and values) using data generated by faker. For each key/value pair a faker object is to be instantiated once. Key is a test input value, value expected result. Both key and value are expressed using some mini language. Like, 's' is to be replaced with a sentence. So no, no room for dict comprehension. – x-yuri Jun 23 '19 at 23:22
  • @x-yuri sure there is. However, it sounds like you overloaded a dictionary with too much meaning anyway. – Martijn Pieters Jun 24 '19 at 00:35
  • I've got a function that transforms double newlines into paragraphs, like `s\n\ns` -> `

    s

    s`, `s\r\rs` -> `

    s

    s`, and so on. Where `s` is a sentence. How do I "unload" the dictionary, or how do I use dictionary comprehension when the key and the value must have access to the same random values (sentences used in this test)?

     – x-yuri Jun 24 '19 at 07:24
  • @x-yuri: this is getting way out of hand for comments. `processed = map(nl_to_para, faked_data)`, `result = {key_expr: value_expr for text in processed}`. You can put the `map()` call in the dict comprehension (replace `in processed` with `in map(...)`). You can replace the `map()` with a generator expression. Etc. Iteration is extensible. – Martijn Pieters Jun 24 '19 at 11:11
  • I hope showing [the code](https://gist.github.com/x-yuri/55614b444756b857fbb2078009f83284) will clarify the issue. I don't see a way to avoid using `map`. But if there is one, I bet it's harder to understand. Then, I'm not sure how people usually go about this, but here's a link to a room to discuss it further (if need be): https://chat.stackoverflow.com/rooms/195480/room-for-x-yuri-and-martijn-pieters – x-yuri Jun 24 '19 at 21:14
41

These toolz are great for this kind of simple yet repetitive logic.

http://toolz.readthedocs.org/en/latest/api.html#toolz.dicttoolz.valmap

Gets you right where you want to be.

import toolz
def f(x):
  return x+1

toolz.valmap(f, my_list)
JTE
  • 1,301
  • 12
  • 14
32

Due to PEP-0469 which renamed iteritems() to items() and PEP-3113 which removed Tuple parameter unpacking, in Python 3.x you should write Martijn Pieters♦ answer like this:

my_dictionary = dict(map(lambda item: (item[0], f(item[1])), my_dictionary.items()))
nathancahill
  • 10,452
  • 9
  • 51
  • 91
lucidyan
  • 3,575
  • 2
  • 22
  • 24
28

You can do this in-place, rather than create a new dict, which may be preferable for large dictionaries (if you do not need a copy).

def mutate_dict(f,d):
    for k, v in d.iteritems():
        d[k] = f(v)

my_dictionary = {'a':1, 'b':2}
mutate_dict(lambda x: x+1, my_dictionary)

results in my_dictionary containing:

{'a': 2, 'b': 3}
Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
gens
  • 972
  • 11
  • 22
  • 3
    Cool, you should maybe rename `mapdict` to `mutate_values_with` or something to make it crystal clear that you rewrite the dict! :) – Tarrasch Aug 24 '14 at 17:10
  • 2
    `zip(d.keys(), d.values())` works for more versions instead of `iteritems()` – rassa45 Aug 07 '15 at 09:29
  • Also, instead of the for loop you could do something similar to what was done in an earlier comment in another answer, `dict(zip(d.keys(), [f(v) for v in dict.values()]))` – rassa45 Aug 07 '15 at 09:31
  • Or dict comprehension (not sure if it works on all versions) `{k:f(v) for k,v in zip(d.keys(), d.values())}` – rassa45 Aug 07 '15 at 09:32
  • 1
    @ytpillai 'zip' or comprehensions make a copy, rather than changing the values in-place, which is the purpose of my answer. The accepted answer is the best one for when a copy is ok. – gens Aug 10 '15 at 15:42
  • 1
    My apologies, I didn't realize you wanted to use the items method. However, another improvement to that is possible as well (for non Python 2.7 users) `{k:f(v) for k,v in iter(d.items())}` – rassa45 Aug 10 '15 at 17:57
  • 1
    Saves space by making an iterator – rassa45 Aug 10 '15 at 17:57
4

While my original answer missed the point (by trying to solve this problem with the solution to Accessing key in factory of defaultdict), I have reworked it to propose an actual solution to the present question.

Here it is:

class walkableDict(dict):
  def walk(self, callback):
    try:
      for key in self:
        self[key] = callback(self[key])
    except TypeError:
      return False
    return True

Usage:

>>> d = walkableDict({ k1: v1, k2: v2 ... })
>>> d.walk(f)

The idea is to subclass the original dict to give it the desired functionality: "mapping" a function over all the values.

The plus point is that this dictionary can be used to store the original data as if it was a dict, while transforming any data on request with a callback.

Of course, feel free to name the class and the function the way you want (the name chosen in this answer is inspired by PHP's array_walk() function).

Note: Neither the try-except block nor the return statements are mandatory for the functionality, they are there to further mimic the behavior of the PHP's array_walk.

Community
  • 1
  • 1
7heo.tk
  • 1,074
  • 12
  • 23
  • 1
    This fails to solve the OP question since the `__missing__` method won't be called for existing keys, which we want to transform, unless the factory method passed use the origin dict as a fallback somehow, but as that is not part of the example usage, I consider this an unsatisfactory answer to the problem at hand. – Kaos Feb 10 '16 at 14:11
  • Which existing keys? – 7heo.tk May 19 '16 at 12:40
  • From the OP: `Given a dictionary { k1: v1, k2: v2 ... } ...`. That is, you already have a `dict` to begin with.. – Kaos May 19 '16 at 21:23
  • I would like to say that we're both right; but I believe that we're both wrong. You're right in that my answer doesn't answer the question; but not for the reason you invoked. I simply missed the point, giving a way to obtain `{v1: f(v1), v2: f(v2), ...}` given `[v1, v2, ...]`, and not given a dict. I will edit my answer to correct that. – 7heo.tk May 20 '16 at 06:34
4

To avoid doing indexing from inside lambda, like:

rval = dict(map(lambda kv : (kv[0], ' '.join(kv[1])), rval.iteritems()))

You can also do:

rval = dict(map(lambda(k,v) : (k, ' '.join(v)), rval.iteritems()))
yourstruly
  • 972
  • 1
  • 9
  • 17
  • 1
    That’s a clever manipulation within the 2-tuple itself in the second example. However, it utilizes auto tuple unpacking within the lambda, which is no longer supported in Python 3. Therefore `lambda(k,v)` will not work. See https://stackoverflow.com/questions/21892989/what-is-the-good-python3-equivalent-for-auto-tuple-unpacking-in-lambda – Jonathan Komar Dec 04 '19 at 09:32
1

Just came accross this use case. I implemented gens's answer, adding a recursive approach for handling values that are also dicts:

def mutate_dict_in_place(f, d):
    for k, v in d.iteritems():
        if isinstance(v, dict):
            mutate_dict_in_place(f, v)
        else:
            d[k] = f(v)

# Exemple handy usage
def utf8_everywhere(d):
    mutate_dict_in_place((
        lambda value:
            value.decode('utf-8')
            if isinstance(value, bytes)
            else value
        ),
        d
    )

my_dict = {'a': b'byte1', 'b': {'c': b'byte2', 'd': b'byte3'}}
utf8_everywhere(my_dict)
print(my_dict)

This can be useful when dealing with json or yaml files that encode strings as bytes in Python 2

Oyono
  • 377
  • 1
  • 8
0
  • My way to map over dictionary
def f(x): return x+2
bill = {"Alice": 20, "Bob": 10}
d = {map(lambda x: f(x),bill.values())}
print('d: ',dict(d))

Results

: d:  {22: 12}
  • Map over iterable in values within dictionary
bills = {"Alice": [20, 15, 30], "Bob": [10, 35]}
d= {map(lambda v: sum(v),bills.values())}
g= dict(map(lambda v: (v[0],sum(v[1])),bills.items()))
# prints
print('d: ',dict(d))
print('g: ',g)

Results

d:  {65: 45}
g:  {'Alice': 65, 'Bob': 45}
teuffy
  • 1
  • 1