Organizing large Data in lists - Structure?

Question

I am dealing with tracking data (x,y,z) with about 180 000 entries per set. The amount of sets is not fixed, but around 22 - 30.

So in the worst case, I have to deal with 5.5 million entries.

Right now, the amount of sets was fixed, so I used a List for each set.

public class TrackingCoords
{
    public int id,
        m,
        n;
    public decimal x,
        y,
        z,
        s;
    public string timekey;

}

public static class Data {
  public static List<TrackingCoords> lstcoords1 = new List<TrackingCoords>();
...
}

How would you organize a dynamic amount of sets without Performance decreases? Can the data be expressed in some kind of nested List?

Use [properties](http://stackoverflow.com/q/295104/1997232). And about what kind of performance decreases you are talking about? Search? Insert? — Sinatr, Dec 18 '15 at 12:41
could you give me an ecample with properties? Performance Decreases in searching the lists. Ideally i need to find an entry from each list every 0.04s (25 frames per second) — ManuKaracho, Dec 18 '15 at 12:43
Possible duplicate of [What .NET collection provides the fastest search](http://stackoverflow.com/questions/1009107/what-net-collection-provides-the-fastest-search) — Sinatr, Dec 18 '15 at 12:44
How would I call an explicit member of the HashSet? hashset["myList1"].x ? — ManuKaracho, Dec 18 '15 at 12:52
See [this](http://stackoverflow.com/q/7760364/1997232). I don't recommend to use `HashSet`, it's only good to check for duplicates. To search for partial data match you would need to hash each individually. If you want to quickly search by id, then `Dictionary` will do (note what `id` appears twice as member of data and as key). For multiple searches you can use nested dictionaries, e.g. `Dictionary>>` (not sure if it's a good structure, but it's pretty fast, unless you can combine `xyz` into just one key, then that is faster). — Sinatr, Dec 18 '15 at 13:09
It depends on what you're searching for. Are you searching by `timekey`, or by some of those cryptically named values (`x`,`m`,`n`, etc.)? — Jim Mischel, Dec 18 '15 at 15:21
Are the sets static (i.e. not changing) throughout the program? Or are you continually updating the lists? Also, when you search, do you want to search individual sets or all sets? Can there be multiple entries with the same `timekey`? With the same `n`? When you get a match, do you want to report which set it came from? You need to give us much more information. — Jim Mischel, Dec 18 '15 at 16:41

score 0 · Answer 1 · answered Dec 18 '15 at 12:59

In list,time complexity of searching is O(n).Using vector will be better,it can use binary-search so time complexity is O(logn).I think the best choose is to use HashSet(System.Collections.Generic.HashSet).You can see more about it in https://msdn.microsoft.com/zh-cn/library/bb359438.aspx

Organizing large Data in lists - Structure?

1 Answers1