Search and remove Duplicates in a List

Question

In this post Other Post I used the programmers suggestion of List<KeyValuePair<string, string>> IdentityLines = new List<KeyValuePair<string, string>>(); to collect multiple string values within certain files of a directory. I now want to remove the duplicate values from that list. Any idea how I can do that in C#? Thanks

See also http://stackoverflow.com/questions/47752/remove-duplicates-from-a-listt-in-c — Robert Harvey, Aug 10 '10 at 20:55

score 5 · Answer 1 · answered Aug 10 '10 at 20:46

5

Use the Distinct method that is found with Linq. Here is an example using a list of ints.

Using System.Linq;

List<int> list = new List<int> { 1, 2, 3, 1, 3, 5 };
List<int> distinctList = list.Distinct().ToList();

answered Aug 10 '10 at 20:46

Jerod Houghtelling

4,783
1
22
30

Robert Harvey · Accepted Answer · 2010-08-10T21:02:22.553

2

static List<T> RemoveDuplicates<T>(List<T> inputList) 
{ 
    Dictionary<T, int> uniqueStore = new Dictionary<T, int>(); 
    List<T> finalList = new List<T>(); 

    foreach (string currValue in inputList) 
    { 
        if (!uniqueStore.ContainsKey(currValue)) 
        { 
            uniqueStore.Add(currValue, 0); 
            finalList.Add(currValue); 
        } 
    } 
    return finalList; 
}

http://www.kirupa.com/net/removingDuplicates.htm

If you want to return an IEnumerable instead, change your return type to IEnumerable<T> and yield return currValue instead of adding it to the final list.

edited Aug 10 '10 at 21:02

answered Aug 10 '10 at 20:47

Robert Harvey

178,213
47
333
501

On that link you posted after this line List result = removeDuplicates(input); What would the foreach loop look like if I wanted to write to a text file? Here is my guess but I get an error on foreach stating cannot convert type string to System.Collections.Generic.List foreach (List lines in result) { StreamWriter fs = File.AppendText(@"C:\Logs\" + "UserSummary" + ".log"); fs.Write(lines.Value + "\r\n"); fs.Close(); } – Josh Aug 10 '10 at 21:02
I think you want `foreach(string s in result)`, based on the error you provided. – Robert Harvey Aug 10 '10 at 21:05
Hi Robert- If I do that then how do I write the value from s? s doesn't have s.value that I can use at fs.Write(); – Josh Aug 10 '10 at 21:07
Well, I can't see you're code, but it's definitely *not* `foreach(list lines in result)`. It *might* be `foreach(KeyValuePair p in result)` – Robert Harvey Aug 10 '10 at 21:10
Hi Robert- Your last post is close but throws an error on foreach stating connot convert type string to Systems.Collections.Generic.KeyValuePair – Josh Aug 10 '10 at 21:16
1

Ah, see it still expects a string, which means that the list you are providing it (the `result` variable) contains strings, not key value pairs. Check your code carefully. – Robert Harvey Aug 10 '10 at 21:23

score 0 · Answer 3 · answered Feb 06 '14 at 23:50

I know this an old question, but here's how I do this:

var inputKeys = new List<KeyValuePair<string, string>>
                          {
                              new KeyValuePair<string, string>("myFirstKey", "one"),
                              new KeyValuePair<string, string>("myFirstKey", "two"),
                              new KeyValuePair<string, string>("mySecondKey", "one"),
                              new KeyValuePair<string, string>("mySecondKey", "two"),
                              new KeyValuePair<string, string>("mySecondKey", "two"),
                          };
 var uniqueKeys = new List<KeyValuePair<string, string>>();

 //get rid of any duplicates
 uniqueKeys.AddRange(inputKeys.Where(keyPair => !uniqueKeys.Contains(keyPair)));

 Assert.AreEqual(inputKeys.Count(), 5);
 Assert.AreEqual(uniqueKeys.Count(), 4);

Search and remove Duplicates in a List

3 Answers3