Trying to find the first 80 unique elements of an object

Question

I am trying to get the first 80 unique elements of an array of objects. My data has about 1500 records https://pastebin.com/knT3eYyx So far I can get the first 100 elements of the array after duplicates, if any. But I want to be able to do this for the whole array Restarting every time it has found 80 unique elements

  const result = Array.from(new Set(DATA.map((a) => a.Numbers))).map(
    (Numbers) => {
      return DATA.find((a) => a.Numbers === Numbers);
    }
  );

Yes I want a final array with 80 items with the Numbers field having no duplicates. And want to be able to perform the function after on the remainder of the original array. — Shiva, Sep 04 '21 at 01:36
Note: [this was already posted](https://stackoverflow.com/questions/69047346/deduplicate-array-of-objects-by-given-property-name/69047569#69047569) but under a different question. — Andy, Sep 04 '21 at 01:41
`const res = Array.from(new Set(DATA.map((a) => a.Numbers))).slice(0, 80).map( (Numbers) => { return DATA.find((a) => a.Numbers === Numbers); } );` — navnath, Sep 04 '21 at 01:42
Correct but the question did not answer my final goal. Thank you will try with the above solution — Shiva, Sep 04 '21 at 01:45
You mean you don't want to compute every time if you want 80 first time and 100 next time? — navnath, Sep 04 '21 at 01:51
Actually the intention is to grab the first 80 unique numbers put in a new array for use, and discard those 80 numbers from the original array so I can re run the function. Basically shrinking the array each time by 80 — Shiva, Sep 04 '21 at 01:54
While discarding those 80 elements do you want to remove their duplicate also? — navnath, Sep 04 '21 at 02:25
Yes. I basically always want to have 80 unique numbers at a given time so I can compare it to the 20 numbers that are not there. But once the first 80 numbers have been analyzed they need to be removed from the array so I can continue with the next set of numbers — Shiva, Sep 04 '21 at 02:38
Check with your large dataset `https://jsfiddle.net/navnathjadhav/zykrabvd/1/` just pass 80 to getNextUniqueRecords method — navnath, Sep 04 '21 at 02:47
This works perfectly! Thank you so much. How can I get this to remove also those duplicates from the original array. It currently only has removed the unique ones. @NavnathJadhav — Shiva, Sep 04 '21 at 18:11
According to the whole object, do you want to remove duplicates? At what stage do you want to remove all duplicates from the original array? At the first stage itself, before getting the first 80 unique numbers? I have modified the previous solution. Added `removeDuplicatesByProperty('Time')` you can pass your required property here and also added `removeDuplicateObjects()` `https://jsfiddle.net/navnathjadhav/zykrabvd/29/` — navnath, Sep 04 '21 at 22:37
So I want to get 80 Unique numbers without any duplicates. The original array should remove all the duplicates plus the 80 so basically I can start counting from the last number that was unique. In this case if I run the function and pass 80 last index is dated '2/10/20'. Everything up to that point should be removed. — Shiva, Sep 05 '21 at 00:39
@NavnathJadhav I don't ever want to remove all duplicates at once I always want to do it in increments of 80. If all duplicates are removed then we would end up being just 100 numbers. Every time a new set of 80 unique numbers have been found everything up to that point needs to be eliminated — Shiva, Sep 05 '21 at 01:13
You want to remove everything from where you start iterating on DATA till you get unique 80 records (by Number)? If so `https://jsfiddle.net/navnathjadhav/zykrabvd/84/` — navnath, Sep 05 '21 at 02:44
When I put it on my file I get an error on line "DATA = DATA.slice(i + 1)" Uncaught TypeError: Assignment to constant variable. @NavnathJadhav not sure why I am importing the data with simple import / export and the data I have changed from a const to a let — Shiva, Sep 05 '21 at 04:35
For while please check if above code works by assigning data = DATA. and replace DATA with data. share import/export code. — navnath, Sep 05 '21 at 04:46
Can you clarify the case, adding missing key information to the question that has been discussed in the comments section? This will improve your question's rating, because it's valuable in the essence, but lacks in the form. — Dawid Pura, Sep 08 '21 at 07:58

score 0 · Answer 1 · answered Sep 05 '21 at 13:23

    let DATA = [
      {
        Date: '8/28/21',
        Time: 'M',
        Numbers: 50,
      },
      {
        Date: '8/28/21',
        Time: 'M',
        Numbers: 50,
      },
      {
        Date: '8/28/21',
        Time: 'F',
        Numbers: 50,
      },
      {
        Date: '8/29/21',
        Time: 'E',
        Numbers: 81,
      },
      {
        Date: '8/29/21',
        Time: 'E',
        Numbers: 81,
      },
      {
        Date: '8/29/21',
        Time: 'M',
        Numbers: 45,
      },
      {
        Date: '8/29/21',
        Time: 'M',
        Numbers: 45,
      },
      {
        Date: '8/28/21',
        Time: 'M',
        Numbers: 50,
      },
    ];

    function getNextUniqueRecordsByNumber(no){
        const numbers = new Set();
        const result = [];
        for(let i=0; i<DATA.length; i++){
            const item = DATA[i];
            if(!numbers.has(item.Numbers)){
                result.push(item);
                numbers.add(item.Numbers);
            }
        if(result.length==no){
            DATA = DATA.slice(i+1);
          break;
        }
        }
      return result;
    }
    
    console.log('before', DATA);
    console.log('res : ',getNextUniqueRecordsByNumber(3));
    console.log('afer', DATA);
    console.log("================")
    console.log('res : ',getNextUniqueRecordsByNumber(2));
    console.log('afer', DATA);
    console.log("================")
    console.log('res : ',getNextUniqueRecordsByNumber(2));

Trying to find the first 80 unique elements of an object

1 Answers1