How to get distinct elements(case insensitive) from array of objects using lodash

Question

I have an array that looks something like below,

[
  {
    "_id": "5e3ccb88c9b3027ef4977894",
    "name": "microsoft"
  },
  {
    "_id": "59ce020caa87df4da0ee2c77",
    "name": "Microsoft"
  },
  {
    "_id": "5e077c78bc0d663d7170ba1c",
    "name": "MICROSOFT"
  },
  {
    "_id": "608839e8d9271457814a7aa4",
    "name": "Microsoft "
  },
  {
    "_id": "5ecd46657ffa9b761a0e41cd",
    "name": "Microsoft  - MSN"
  },
  {
    "_id": "5dfb47adbc0d663d716fe25f",
    "name": "Microsoft  Alumni Foundation"
  }
]

now I need only one Microsoft, instead of these case combinations using lodash.

expected output,

[
  {
    "_id": "608839e8d9271457814a7aa4",
    "name": "Microsoft "
  },
  {
    "_id": "5ecd46657ffa9b761a0e41cd",
    "name": "Microsoft  - MSN"
  },
  {
    "_id": "5dfb47adbc0d663d716fe25f",
    "name": "Microsoft  Alumni Foundation"
  }
]

can anyone help me out with this? found many solution, but i need the case insensitive filter.

thanks in advance.

Is there are criteria for which `'Microsoft ...'` you choose, or do you only want to select the first one you see? — lejlun, Sep 03 '21 at 10:19
No @lejlun, I just don't need the duplicate values, that might be in lower or upper case. I just want unique names. — Nick, Sep 03 '21 at 10:22
Does this answer your question? [How to remove all duplicates from an array of objects?](https://stackoverflow.com/questions/2218999/how-to-remove-all-duplicates-from-an-array-of-objects) — Tyler2P, Sep 03 '21 at 10:24
This is a good one, but didn't find any way to do that case insensitive filter. — Nick, Sep 03 '21 at 10:29

score 2 · Answer 1 · answered Sep 03 '21 at 10:48

You can use _.filter, _.toLower and _.trim to do this.

We can keep a seen Set() of values, so we know which items we've seen before. And using _.toLower and _.trim before adding or checking this set we can make sure that the item is not a slight variation of another item in the array.

Like this:

let data = [ { _id: '5e3ccb88c9b3027ef4977894', name: 'microsoft' }, { _id: '59ce020caa87df4da0ee2c77', name: 'Microsoft' }, { _id: '5e077c78bc0d663d7170ba1c', name: 'MICROSOFT' }, { _id: '608839e8d9271457814a7aa4', name: 'Microsoft ' }, { _id: '5ecd46657ffa9b761a0e41cd', name: 'Microsoft  - MSN' }, { _id: '5dfb47adbc0d663d716fe25f', name: 'Microsoft  Alumni Foundation' }, ];

const testPrefixes = new Set();

let res = _.filter(data, (o) => {
  if (testPrefixes.has(_.trim(_.toLower(o.name)))) return false;
  else return testPrefixes.add(_.trim(_.toLower(o.name)));
});

console.log(res);

<script src="https://cdn.jsdelivr.net/npm/lodash@4.17.21/lodash.min.js"></script>

This can all be done without `lodash`, it might even be faster. Is there a specific reason that `lodash` is required? — lejlun, Sep 03 '21 at 10:50

Ori Drori · Accepted Answer · 2021-09-03T17:14:26.243

You can use lodash's _.uniqBy(), which accepts a function that generates the matching criteria. In this case trim the string, and convert it to lower case:

const arr = [{"_id":"5e3ccb88c9b3027ef4977894","name":"microsoft"},{"_id":"59ce020caa87df4da0ee2c77","name":"Microsoft"},{"_id":"5e077c78bc0d663d7170ba1c","name":"MICROSOFT"},{"_id":"608839e8d9271457814a7aa4","name":"Microsoft "},{"_id":"5ecd46657ffa9b761a0e41cd","name":"Microsoft  - MSN"},{"_id":"5dfb47adbc0d663d716fe25f","name":"Microsoft  Alumni Foundation"}]

const result = _.uniqBy(arr, o => o.name.trim().toLowerCase())

console.log(result)

<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.21/lodash.min.js" integrity="sha512-WFN04846sdKMIP5LKNphMaWzU7YpMyCU245etK3g/2ARYbPK9Ub18eG+ljU96qKRCWh+quCY7yefSmlkQw1ANQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>

Since you want the last item in a series of duplicates, you can reverse the array, get the unique items, and then reverse it back:

const arr = [{"_id":"5e3ccb88c9b3027ef4977894","name":"microsoft"},{"_id":"59ce020caa87df4da0ee2c77","name":"Microsoft"},{"_id":"5e077c78bc0d663d7170ba1c","name":"MICROSOFT"},{"_id":"608839e8d9271457814a7aa4","name":"Microsoft "},{"_id":"5ecd46657ffa9b761a0e41cd","name":"Microsoft  - MSN"},{"_id":"5dfb47adbc0d663d716fe25f","name":"Microsoft  Alumni Foundation"}]

const result = _.reverse(_.uniqBy(
  _.reverse([...arr]), 
  o => o.name.trim().toLowerCase()
))

console.log(result)

<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.21/lodash.min.js" integrity="sha512-WFN04846sdKMIP5LKNphMaWzU7YpMyCU245etK3g/2ARYbPK9Ub18eG+ljU96qKRCWh+quCY7yefSmlkQw1ANQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>

Seems like the most concise way of expressing in lodash... however `uniqBy` appears to choose the first of the duplicated items, whereas OP's desired output suggests they want the last of the duplicated items. — spender, Sep 03 '21 at 16:07
Yep. Missed the last duplicate part. I've added a solution for that as well, although now it's less concise. — Ori Drori, Sep 03 '21 at 17:15

spender · Answer 3 · 2021-09-03T11:24:48.007

Here's a library-free utility (in TypeScript) that I keep to hand to do exactly this:

const dedupeArrayBy = <T, TK>(items: T[], selector: (item: T) => TK): T[] => {
    const m = new Map(items.map((item) => [selector(item), item] as const));
    return [...m.values()];
};

so you can:

const deduped = dedupeArrayBy(items, v => v.name.trim().toLowerCase());

Here is the Javascript translation:

"use strict";
const dedupeArrayBy = (items, selector) => {
  const m = new Map(items.map((item) => [selector(item), item]));
  return [...m.values()];
};
let data = [{
  _id: '5e3ccb88c9b3027ef4977894',
  name: 'microsoft'
}, {
  _id: '59ce020caa87df4da0ee2c77',
  name: 'Microsoft'
}, {
  _id: '5e077c78bc0d663d7170ba1c',
  name: 'MICROSOFT'
}, {
  _id: '608839e8d9271457814a7aa4',
  name: 'Microsoft '
}, {
  _id: '5ecd46657ffa9b761a0e41cd',
  name: 'Microsoft  - MSN'
}, {
  _id: '5dfb47adbc0d663d716fe25f',
  name: 'Microsoft  Alumni Foundation'
}, ];
console.log(dedupeArrayBy(data, v => v.name.trim().toLowerCase()));

How to get distinct elements(case insensitive) from array of objects using lodash

3 Answers3