Linq Distinct not being very Distinct

Question

Is this linq statement correct for getting unique values of 'Name' from my newSpecs collection?

distinctSpecialties is showing duplicates.

IEnumerable<SpecialtyData> distinctSpecialties = newSpecs
    .Select(s => s).Distinct()
    .OrderBy(s => s.Name);

Answer 1

You need to define "distinct" - by default reference types are distinct if they are separate instances (ReferenceEquals). You can override the default by either:

• Implementing IEquatable<SpecialtyData> in SpecialtyData
• Overriding Equals and GetHashCode in SpecialtyData
• defining an IEqualityComparer<SpecialtyData> and passing that to Distinct

A simpler way to get the "first" distinct item by name is to use GroupBy:

IEnumerable<SpecialtyData> distinctSpecialties = newSpecs.GroupBy(s => s.Name)
                                                         .Select(g => g.First());

Answer 2

You're asking about getting unique values of 'Name' from my newSpecs collection, but your code returns IEnumerable<SpecialtyData>. Something is wrong: either the question or the return type.

If you need only names, you need IEnumerable<string> instead:

IEnumerable<string> distinctNames = newSpecs
    .Select(s => s.Name)
    .Distinct().OrderBy(x => x);

If you need entire SpecialtyData instances, you can either use GroupBy+First:

IEnumerable<SpecialtyData> distinctSpecialties = newSpecs
    .GroupBy(s => s.Name)
    .Select(g=> g.First())

or override Equals and GetHashCode methods within SpecialtyData class to make Distinct work.

Answer 3

That's because the default comparison is a reference comparison. It's not looking at the data itself, it's giving you distinct references. Either use your Select() to get some value worth comparing or implement IEqualityComparer<T> for your type and pass it to Distinct.

Below is a sample implementation of IEqualityComparer<T>;

class SpecialtyDataEqualityComparer : IEqualityComparer<SpecialtyData>
{
    public bool Equals(SpecialtyData lhs, SpecialtyData rhs)
    {
        return lhs.Name == rhs.Name;
    }

    public int GetHashCode(SpecialtyData p)
    {
        return p.Name.GetHashCode();
    }
}

If you did .Distinct(new SpecialtyDataEqualitComparer()) then you would get items with distinct names. If you want some other definition of equality then change the logic in Equals to do the comparisons you require (make related changes to GetHashCode so they are consistent, if you do equality by name you shouldn't take the hashcode for some other property).

Answer 4

Instead of doing custom equality comparers and all that, there is a really nice extension method from MoreLinq - https://code.google.com/p/morelinq/ (by Jon Skeet).

Either download the whole library of MoreLinq, or just add this code below:

public static IEnumerable<TSource> DistinctBy<TSource, TKey>(
                            this IEnumerable<TSource> source, Func<TSource, TKey> selector)
{
    var set = new HashSet<TKey>();
    return source.Where(element => set.Add(selector(element)));
}

And use it like so:

IEnumerable<SpecialtyData> distinctSpecialties = newSpecs
    .Select(s => s).DistinctBy(s => s.Name)
    .OrderBy(s => s.Name);

Answer 5

You can use GroupBy as below

IEnumerable<SpecialtyData> distinctSpecialties = 
               newSpecs.GroupBy(s => s.Name).Select(g=> g.First()).OrderBy(s => s.Name);

Answer 6

In order to use Distinct() on a custom object (SpecialtyData), you will need to override the two methods Distinct() uses to determine equality.

    public override bool Equals(object other) {

    }

    public override int GetHashCode() {

    } 

Each of these two should return values that define the SpecialtyData objects as unique. In your case it may be as simple as evaluating the 'Name' property.

Answer 7

There's a very nice post from James Michael Hare that explains why you're getting this results, and I'll quote it here:

For Distinct() (and many other LINQ features) to work, the class being compared (SpecialtyData in your example) must implement Equals() and GetHashCode(), or alternatively provide a separate IEqualityComparer<T> as an argument to Distinct().

Many LINQ methods take advantage of GetHashCode() for performance because internally they will use things like a Set<T> to hold the unique items, which uses hashing for O(1) lookups. Also, GetHashCode() can quickly tell you if two objects may be equivalent and which ones are definitely not - as long as GetHashCode() is properly implemented of course.

So you should make all your classes you intend to compare in LINQ implement Equals() and GetHashCode() for completeness, or create a separate IEqualityComparer<T> implementation.

Original post: This code returns distinct values. However, what I want is to return a strongly typed collection as opposed to an anonymous type