Fastest way to select distinct values from list based on two properties

Question

I have a this list:

List<myobject> list= new List<myobject>();

list.Add(new myobject{name="n1",recordNumber=1}); 
list.Add(new myobject{name="n2",recordNumber=2}); 
list.Add(new myobject{name="n3",recordNumber=3});
list.Add(new myobject{name="n4",recordNumber=3});

I'm looking for the fastest way to select distinct objects based on recordNumber, but if there is more than one object with same recordNumber(here recordNumber=3), I want to select object base on its name.(the name provided by paramater)

thanks

Can you clarify what you mean by "...but if there is more than one object with same recordNumber, I want to select object base on its name."? — Enigmativity, Aug 29 '12 at 02:42
sure, if we have to object with same record number(recordNumber=3), I Should select the object that its name is "n3". this vale ("n3") is a paramater and it should be use for all duplicate objects. — sal el, Aug 29 '12 at 02:49
No, public myobject Find(string name) { } we use name parameter for duplicat objects. — sal el, Aug 29 '12 at 03:05
Which would mean that you could miss objects if the duplicate record numbers don't have a name with the passed in parameter? — Enigmativity, Aug 29 '12 at 03:10
yes, with duplicate, I only need the object with a name equals to the parameter. — sal el, Aug 29 '12 at 03:17
What if there are duplicates but none of them have the name passed as the parameter? — Enigmativity, Aug 29 '12 at 03:19

score 2 · Accepted Answer · answered Aug 29 '12 at 02:44

2

It looks like you are really after something like:

Dictionary<int, List<myobject>> myDataStructure;

That allows you to quickly retrieve by record number. If the List<myobject> with that dictionary key contains more than one entry, you can then use the name to select the correct one.

Note that if your list is not terribly long, an O(n) check that just scans the list checking for the recordNumber and name may be fast enough, in the sense that other things happening in your program could obscure the list lookup cost. Consider that possibility before over-optimizing lookup times.

answered Aug 29 '12 at 02:44

Eric J.

147,927
63
340
553

thanks, but unfortunately I have a dll that returns a list of object as I said in question. I’m not sure creating dictionary from list is the fastest way. – sal el Aug 29 '12 at 02:54
@salel: That depends on how frequently you will search for elements vs. how frequently you will rebuild the list, and on how large the list is. Searching an unordered list will always be O(n). – Eric J. Aug 29 '12 at 02:56

score 2 · Answer 2 · answered Aug 29 '12 at 03:18

Here's the LINQ way of doing this:

Func<IEnumerable<myobject>, string, IEnumerable<myobject>> getDistinct =
    (ms, n) =>
        ms
            .ToLookup(x => x.recordNumber)
            .Select(xs => xs.Skip(1).Any()
                ? xs.Where(x => x.name == n).Take(1)
                : xs)
            .SelectMany(x => x)
            .ToArray();

I just tested this with a 1,000,000 randomly created myobject list and it produced the result in 106ms. That should be fast enough for most situations.

thanks, I will use these solutions and check it which one is faster in my environment. — sal el, Aug 29 '12 at 03:30

score 1 · Answer 3 · answered Aug 29 '12 at 03:14

Are you looking for

class Program
    {
        static void Main(string[] args)
        {
            List<myobject> list = new List<myobject>();

            list.Add(new myobject { name = "n1", recordNumber = 1 });
            list.Add(new myobject { name = "n2", recordNumber = 2 });
            list.Add(new myobject { name = "n3", recordNumber = 3 });
            list.Add(new myobject { name = "n4", recordNumber = 3 });

            //Generates Row Number on the fly
            var withRowNumbers = list 
                    .Select((x, index) => new 
                            {
                                Name = x.name,
                                RecordNumber = x.recordNumber,
                                RowNumber = index + 1
                            }).ToList();

            //Generates Row Number with Partition by clause
            var withRowNumbersPartitionBy = withRowNumbers
                    .OrderBy(x => x.RowNumber)
                    .GroupBy(x => x.RecordNumber)
                    .Select(g => new { g, count = g.Count() })
                    .SelectMany(t => t.g.Select(b => b)
                    .Zip(Enumerable.Range(1, t.count), (j, i) => new { Rn = i, j.RecordNumber, j.Name}))
                    .Where(i=>i.Rn == 1)
                    .ToList();
            //print the result
            withRowNumbersPartitionBy.ToList().ForEach(i => Console.WriteLine("Name =  {0}   RecordNumber = {1}", i.Name, i.RecordNumber));

            Console.ReadKey();
        }
    }

    class myobject
    {
        public int recordNumber { get; set; }
        public string name { get; set; }
    }

Result:

Name =  n1   RecordNumber = 1
Name =  n2   RecordNumber = 2
Name =  n3   RecordNumber = 3

score 0 · Answer 4 · answered Aug 29 '12 at 02:51

0

Are you looking for a method to do this?

List<myobject> list= new List<myobject>();

list.Add(new myobject{name="n1",recordNumber=1}); 
list.Add(new myobject{name="n2",recordNumber=2}); 
list.Add(new myobject{name="n3",recordNumber=3});
list.Add(new myobject{name="n4",recordNumber=3});

public myobject Find(int recordNumber, string name)
{
    var matches = list.Where(l => l.recordNumber == recordNumber);

    if (matches.Count() == 1)
        return matches.Single();

    else return matches.Single(m => m.name == name);
}

This will - of course - break if there are multiple matches, or zero matches. You need to write your own edge cases and error handling!

answered Aug 29 '12 at 02:51

Kirk Broadhurst

27,836
16
104
169

Linq does not make things faster. If anything, the extra layer of abstraction can make things ever so slightly slower. – Eric J. Aug 29 '12 at 02:57
not exactly. I don't have the record number, I only have the name, if there are more object with same recordNumber, then I should use name for selecting the object. – sal el Aug 29 '12 at 02:57

score 0 · Answer 5 · edited May 23 '17 at 11:56

If the name and recordNumber combination is guaranteed to be unique then you can always use Hashset.

You can then use RecordNumber and Name to generate the HashCode by using a method described here.

class myobject 
{

     //override GetHashCode
     public override int GetHashCode()
     {
        unchecked // Overflow is fine, just wrap
        {
           int hash = 17;
           // Suitable nullity checks etc, of course :)
           hash = hash * 23 + recordNumber.GetHashCode();
           hash = hash * 23 + name.GetHashCode();
           return hash;
         }
     }
     //override Equals      
}

Fastest way to select distinct values from list based on two properties

5 Answers5