int[,] matrix too large (OutOfMemoryException) but have enough ram to hold it

Question

I'm working on a Levenshtein difference and I'm getting a OutOfMemoryException when handling a matrix of int[11000,11000]. From everything I know this isn't going to take up THAT much memory, yet I still receive the OutOfMemoryException. If my calculations are right it's a little under half a gig, I checked and there is enough ram to support this.

What could be the issue, am I hitting a maximum on the array?

How could I handle large arrays without actually knowing how large they may be? Maybe a memory-mapped file?

UPDATE

Here is the code, the exception is thrown on line 10.

private int LevenshteinDifference(string source, string target)
{
    // Check for degenerate cases.
    int sourceLength = source != null ? source.Length : 0;
    int targetlength = target != null ? target.Length : 0;
    if (sourceLength == 0 && targetlength == 0) return 0;
    if (sourceLength == 0) return targetlength;
    if (targetlength == 0) return sourceLength;
    // Do the calculations.
    int[,] distance = new int[sourceLength + 1, targetlength + 1];
    // Fill first row and column of the matrix with consecutive integers starting at 0 position.
    for (int x = 0; x <= sourceLength; x++) distance[x, 0] = x;
    for (int y = 0; y <= targetlength; y++) distance[0, y] = y;
    // We must fill in the remaining cells.
    for (int x = 1; x <= sourceLength; x++)
    {
        for (int y = 1; y <= targetlength; y++)
        {
            // Cost is equal to 0 if x and y match, otherwise 1.
            int cost = source[x - 1] == target[y - 1] ? 0 : 1;
            // min(min(left+1, top+1), diagonal+cost)
            distance[x, y] = Math.Min(Math.Min(distance[x - 1, y] + 1, distance[x, y - 1] + 1), distance[x - 1, y - 1] + cost);
        }
    }
    // Return the bottom-right most cell value. This is the total cost to effectively make source equal the target.
    return distance[sourceLength, targetlength];
}

Do you only have *one* instance of that array? Are you passing it into functions? This is hard to answer without being able to see your code. — , Mar 05 '17 at 18:39
It's set for any CPU but runs under a 64-bit IIS setup. And the array is created in the function, it is not passed nor do I have multiple matrix of this size, just the one. Will share code. — David Carrigan, Mar 05 '17 at 18:54
is "Prefer 32 bit" unchecked ? http://stackoverflow.com/questions/12066638/what-is-the-purpose-of-the-prefer-32-bit-setting-in-visual-studio-2012-and-how — Slai, Mar 05 '17 at 19:02
https://blogs.msdn.microsoft.com/ericlippert/2009/06/08/out-of-memory-does-not-refer-to-physical-memory/ — , Mar 05 '17 at 19:05
You can never run out of RAM. An app never fails because it runs out of memory. You get OOM when there is no longer a hole left in the address space that is big enough to fit the array. You can only ever get a 462 megabytes hole right after your program starts up, the odds rapidly dwindle after that as the address space gets fragmented. Target AnyCPU to get really big holes. — Hans Passant, Mar 05 '17 at 19:18

int[,] matrix too large (OutOfMemoryException) but have enough ram to hold it

0 Answers0