Merging two dicts in python with no duplication permitted

Question

This question differs from similar dictionary merge questions in that conflicting duplicates should fail, or return False. Other solutions use a precedence rule to decide how to manage when one key might be mapped to two different variables.

How do I merge two dicts efficiently in python. As an example, consider:

d1 = {'x': 'a', 'y': 'b', 'z': 'c'}
d2 = {'z': 'c', 'w': 'r'}
d3 = {'z': 'd', 'w': 'r'}

so, the result of merging dictionary 1 and 2 would be

{'x': 'a', 'y': 'b', 'z': 'c', 'w': 'r'}

but the merge of 1 and 3 or 2 and 3 should fail because z has a conflict.

My solution is:

def merge_dicts(d1,d2):
   k1=d1.keys()
   k2=d2.keys()
   unified_dict=dict()
   for k in k1:
       # look up in second dictionary
      if k in k2:
         pt=d2[k]  #pt stands for 'plain text'
         # if lookup is a contradiction, return empty dictionary
         #  don't even bother with partial results
         if pt!=d1[k]:
             return dict()
         else:
             unified_dict[k]=d1[k]  # safe: key is consistent
      else:
          unified_dict[k]=d1[k] # safe:  no key in k2

# get the rest
# already resolved intersection issues so just get set difference
   for k in d2.keys():
      if k not in d1.keys():
          unified_dict[k]=d2[k]

   return unified_dict

Any improvements?

Answer 1

Use dictionary views here; they let you treat dictionary keys as sets:

def merge_dicts(d1, d2):
    try:
        # Python 2
        intersection = d1.viewkeys() & d2
    except AttributeError:
        intersection = d1.keys() & d2
       
    if any(d1[shared] != d2[shared] for shared in intersection):
        return {}  # empty result if there are conflicts

    # leave the rest to C code, execute a fast merge using dict()
    return dict(d1, **d2)

The above code only tests for shared keys referencing non-matching values; the merge itself is best just left to the dict() function.

I made the function work both on Python 2 and Python 3; if you only need to support one or the other, remove the try..except and replace intersection with the relevant expression. In Python 3 the dict.keys() method returns a dictionary view by default. Also, in Python 3-only code I’d use {**d1, **d2} expansion, which is a little faster, cleaner and is not limited to string keys only.

You could conceivably make this a one-liner; Python 3 version:

def merge_dicts(d1, d2):
    return (
        {} if any(d1[k] != d2[k] for k in d1.keys() & d2)
        else {**d1, **d2}
    )

If all you need to support is Python 3.9 or newer, you can use the | dictionary merge operator:

def merge_dicts(d1, d2):
   return (
       {} if any(d1[k] != d2[k] for k in d1.keys() & d2)
       else d1 | d2
   )

Demo:

>>> d1 = {'x': 'a', 'y': 'b', 'z': 'c'}
>>> d2 = {'z': 'c', 'w': 'r'}
>>> d3 = {'z': 'd', 'w': 'r'}
>>> merge_dicts(d1, d2)
{'y': 'b', 'x': 'a', 'z': 'c', 'w': 'r'}
>>> merge_dicts(d1, d3)
{}
>>> merge_dicts(d2, d3)
{}

Answer 2

d1 = {'x': 'a', 'y': 'b', 'z': 'c'}                                                             
d2 = {'z': 'c', 'w': 'r'}
d3 = {'z': 'd', 'w': 'r'}

def dict_merge(d1, d2):
    """docstring for merge"""
    # doesn't work with python 3.x. Use keys(), items() instead
    if len(d1.viewkeys() & d2) != len(d1.viewitems() & d2.viewitems()):
        return {}
    else:
        result = dict(d1, **d2)
        return result

if __name__ == '__main__':
    print dict_merge(d1, d2)

Answer 3

a slightly different approach (pre-check):

d1={'x':'a','y':'b','z':'c'}
d2={'z':'c','w':'r'}
d3={'z':'d','w':'r'}

def merge(d1, d2):
    for (k1,v1) in d1.items():
        if k1 in d2 and v1 != d2[k1]:
            raise ValueError
    ret = d1.copy()
    ret.update(d2)
    return ret

print(merge(d1,d2))
print(merge(d1,d3))

Answer 4

Why not using set ?

#!/usr/bin/python

d1={'x':'a','y':'b','z':'c'}
d2={'w':'r'}
d3={'z':'d','w':'r'}
d4={'x':'a','y':'b','z':'c'}

def merge_dicts(d1, d2):
    dicts = d1.items() + d2.items()
    if len(dicts) != len(set(dicts)):
        raise ValueError
    else:
        return dict(set(dicts))
print merge_dicts(d1, d2)
print merge_dicts(d1, d3)
try:
    print merge_dicts(d1, d4)
except:
    print "Failed"

$ python foo.py
{'y': 'b', 'x': 'a', 'z': 'c', 'w': 'r'}
{'y': 'b', 'x': 'a', 'z': 'd', 'w': 'r'}
Failed

Edit:

Indeed this will not work with not hashable value, this one will:

#!/usr/bin/python
# coding: utf-8 

#!/usr/bin/python

d1={'x':'a','y':'b','z':'c'}
d2={'w':'r'}
d3={'z':'d','w':'r'}
d4={'x':'a','y':'b','z':'c'}

def merge_dicts(d1, d2):
    merged= d1.copy()
    for k, v in d2.iteritems():
        if k in merged:
            raise ValueError
        else:
            merged[k] = v 
    return merged

for one, two in [(d1, d2), (d1, d3), (d1, d4)]:
    try:
        print merge_dicts(one, two)
    except:
        print "Merge Failed for %s with %s" %(one, two)

Answer 5

This does what you want:

def merge_dicts(d1, d2):
    # Join dicts and get rid of non-conflicting dups
    elems = set(d1.items()) | set(d2.items())

    # Construct join dict
    res = {}
    for k, v in elems:
        if k in res.keys():
            return dict()  # conflicting dup found
        res[k] = v;

    return res