combine 2 list of dicts into one

Question

I have two list of dicts say :

rds_detail:

[   {   'rds_name': u'emsclassicldap', 'rds_type': u'db.m3.medium'},
    {   'rds_name': u'fra01-devops-KPX', 'rds_type': u'db.t2.medium'},
    {   'rds_name': u'prodreplica', 'rds_type': u'db.t2.medium'}
]

cloudwatch_detail:

[   {   'alarm_name': u'emsclassicldap_db_connections', 'alarm_threshold': 380.0},
    {   'alarm_name': u'fra01-devops-KPX_db_connection',
        'alarm_threshold': 266.0},
    {   'alarm_name': u'prodreplica_db_connections',
        'alarm_threshold': 266.0},
]

The alarm_name actually has rds_name as its substring ;I need to actually combine these two list into one based this condition so that the final result should look like

[
    {   'rds_name': u'emsclassicldap', 'rds_type': u'db.m3.medium','alarm_name': u'classicldap_db_connections', 'alarm_threshold': 380.0}
    .
    .
    So on
    . 
    .
]

i am writing a simple def to combine :

def combine_rds_cloudwatch(rds_detail,cloudwatch_detail):
    print rds_detail,cloudwatch_detail

    for rds in rds_detail:
        for alarm in cloudwatch_detail:
            if ????????????

Not sure how to do so

I would iterate over the list, get the value of `rds_name` and use `in list2[index]['alarm_name']` do a dict update and merge into a new list — Bobby, Mar 11 '17 at 17:48
You can use the `in` operator to search for substrings, e.g. `if rds['rds_name'] in alarm['alarm_name']:` for that if statement. You'll want to return a new list of dictionaries. Try experimenting with `dict.items()`, which allows you to iterate a dictionary's entries. — Archimaredes, Mar 11 '17 at 17:48

falsetru · Answer 1 · 2017-03-11T17:58:47.933

2

And, dict(one_dict, **other_dict) will give you another dictionary with items from both dictionaries:

>>> d1 = {'a': 1, 'b': 2}
>>> d2 = {'c': 3, 'd': 4}
>>> dict(d1, **d2)  # => dict({'a':1,'b':2}, c=3, d=4)
{'a': 1, 'c': 3, 'b': 2, 'd': 4}

You can use list comprehension (two for clause) to make products and filter them using if clause:

def combine_rds_cloudwatch(rds_detail,cloudwatch_detail):
    return [dict(rds, **alarm) for rds in rds_detail
                               for alarm in cloudwatch_detail
                               if rds['rds_name'] in alarm['alarm_name']]
                          # OR if alarm['alarm_name'].startswith(rds['rds_name'])

edited Mar 11 '17 at 17:58

answered Mar 11 '17 at 17:49

falsetru

357,413
63
732
636

but does it check the condition i mentioned ? rds_name is a substring in alarm_name , it has to check that first – Nishant Singh Mar 11 '17 at 17:53
@NishantSingh, I missed that. I updated the answer accordingly. – falsetru Mar 11 '17 at 17:55
I understand, can u just open up the list comprehension ( i need to understand it , still new here) – Nishant Singh Mar 11 '17 at 17:59
@NishantSingh, I added a link to tutorial (list comprehension). I hope it help you understand. The given list comprehension is basically similar to nested for loop with a if statement. – falsetru Mar 11 '17 at 18:00

mayo · Accepted Answer · 2017-03-11T18:06:27.190

A more generic method..

rds_list = [{'rds_name': u'emsclassicldap', 'rds_type': u'db.m3.medium'},
            {'rds_name': u'fra01-devops-KPX', 'rds_type': u'db.t2.medium'},
            {'rds_name': u'goldenprodreplica', 'rds_type': u'db.t2.medium'}
            ]

cloudwatch_list = [{'alarm_name': u'emsclassicldap_db_connections', 'alarm_threshold': 380.0},
                   {'alarm_name': u'fra01-devops-KPX_db_connection', 'alarm_threshold': 266.0},
                   {'alarm_name': u'goldenprodreplica_db_connections', 'alarm_threshold': 266.0},
                   ]


def merge_two_dicts(x, y):
    """Given two dicts, merge them into a new dict as a shallow copy.
       More info here http://stackoverflow.com/questions/38987/how-to-merge-two-python-dictionaries-in-a-single-expression?rq=1
    """
    z = x.copy()
    z.update(y)
    return z


def combine_list(prefix_key_list, prefix_key, keys_list, key):
    combined_list = []

    for short in prefix_key_list:
        for long in keys_list:
            if long[key].startswith(short[prefix_key]):
                result = merge_two_dicts(long, short)
                combined_list.append(result)

    return combined_list


print(combine_list(rds_list, 'rds_name', cloudwatch_list, 'alarm_name'))

Elmex80s · Answer 3 · 2017-03-11T18:22:51.657

This

from pprint import pprint

pprint([dict(y.items() + next(x.items() for x in rds_detail if x['rds_name'] in y['alarm_name'])) for y in cloudwatch_detail])

Will give

[{'alarm_name': u'emsclassicldap_db_connections', 
  'alarm_threshold': 380.0,
  'rds_name': u'emsclassicldap',
  'rds_type': u'db.m3.medium'},
 {'alarm_name': u'fra01-devops-KPX_db_connection',
  'alarm_threshold': 266.0,
  'rds_name': u'fra01-devops-KPX',
  'rds_type': u'db.t2.medium'},
 {'alarm_name': u'prodreplica_db_connections',
  'alarm_threshold': 266.0,
  'rds_name': u'prodreplica',
  'rds_type': u'db.t2.medium'}]

Is it the correct output?

If you know the order of elements in both lists you can also just do

pprint([dict(x.items() + y.items()) for x, y in zip(rds_detail, cloudwatch_detail)])

Psidom · Answer 4 · 2017-03-11T18:10:09.493

You can use startswith to check if one string starts with another:

from copy import deepcopy
result = []
for rds in rds_detail:
    temp = deepcopy(rds)
    for cw in cloudwatch_detail:
        # replace the condition with rds['rds_name'] in cw['alarm_name'] if the condition 
        # is substring
        if cw['alarm_name'].startswith(rds['rds_name']):
            temp.update(cw)
    result.append(temp)

result
#[{'alarm_name': 'emsclassicldap_db_connections',
#  'alarm_threshold': 380.0,
#  'rds_name': 'emsclassicldap',
#  'rds_type': 'db.m3.medium'},
# {'alarm_name': 'fra01-devops-KPX_db_connection',
#  'alarm_threshold': 266.0,
#  'rds_name': 'fra01-devops-KPX',
#  'rds_type': 'db.t2.medium'},
# {'alarm_name': 'prodreplica_db_connections',
#  'alarm_threshold': 266.0,
#  'rds_name': 'prodreplica',
#  'rds_type': 'db.t2.medium'}]

this is updating in the cw ? i need a seprate list to be returned ,,, so should i just open a new list above the for loop and append ? — Nishant Singh, Mar 11 '17 at 17:57

combine 2 list of dicts into one

4 Answers4