0

I have program where I have to work with extremly big arrays. I need to use two-dimensional arrays.

I've used classical bool[,], but it throwed System.OutOfMemoryException when I tried to declare it when the index was just too big. (Used bool[,] foo = new bool[width + bar, height]). Both numbers width and bar were extremly big, so I suppose, it just exceeded the index.

How can I change index type to Int64 (long)? I know I can use Dictionary<ulong, bool>, but it would force me to rewrite almost all code in my app and I find array much more comfortable to work then with Dictionary, especially when I already have LINQ methods that I need and I don't know, if it would work with two dimensional dictionary.

Thanks, Peter

EDIT 1: Here's the code (note: I've just tested it with sizeX = 2000000000, it threw the exception):

bool[,] pole = new bool[sizeX, maxY];

The problem is, I have many objects with their X, y-length and x-length (Y is always 0). I have an array where on each position, there either is or isn't object tile.

So I just get coord X, width & height and position tiles into array.

The point is, X can be negative, so when I need to position for example 10 objects, I get lowest X (and save it for example largestNegativeX = -999) and before positioning objects into array, I add to every object's X object.X += -largestNegativeX. Then I figure out array width (for example 2000000000) and try to declare the array. And it's where the exception is thrown.

SoptikHa
  • 437
  • 4
  • 19
  • 1
    Use [BitArray](https://msdn.microsoft.com/en-us/library/system.collections.bitarray(v=vs.110).aspx) for that. It stores bool values as bits and consumes less memory. – Yoh Deadfall Oct 08 '17 at 19:16
  • 7
    This looks like it may be an x y problem. You may get a better solution if you ask about your particular use case. – Logarr Oct 08 '17 at 19:20
  • There is nothing wrong with using a dictionary here.. – Chibueze Opata Oct 08 '17 at 19:24
  • 2
    `How can I change index type to Int64 (long)` - you cannot, because [relative addressing is limited to a signed 32-bit offset value](https://stackoverflow.com/a/4267974/11683). Also `OutOfMemoryException` means the array itself was too big, not that the index value was too large, otherwise it would be `OverflowException`. – GSerg Oct 08 '17 at 19:32
  • @Logarr I've extended question. Do you have any idea? – SoptikHa Oct 08 '17 at 19:32
  • `System.OutOfMemoryException` is throwing because you have run out of memory, how is this related to the size of your array. i.e. the `type` which you are using doesn't matter("with a pinch of salt") , what does is how many you are working with at once, try lowering the amount of active object instance. Lets say have only 100 000 instances, and see how much that consumes. – Seabizkit Oct 08 '17 at 19:34
  • OutOfMemoryException means that there's no enough memory to allocate your array. This happens when you're setting size using Int32. What result are you planning to get using Int64? Do you think that this will find available memory somewhere? – Dennis Oct 08 '17 at 19:38
  • Looking at your example you are trying to allocate 2 000 000 000 `bool` values in memory **multiplied `maxY` times**. This means about 1.86 GB of data even if you keep `maxY = 1`. You are using arrays in the worst possible way. – Federico Dipuma Oct 08 '17 at 19:55
  • I believe [linked as duplicate](https://stackoverflow.com/questions/2338778/what-is-the-maximum-length-of-an-array-in-net-on-64-bit-windows) post answers your question *as asked* - no, there is no practical way to use type wider than `int` to index arrays. Since it is unclear what your actual goal is it is hard to offer alternative solution, but look into [sparse arrays](https://stackoverflow.com/questions/3798178/implementing-a-sparse-array-in-c-sharp-fastest-way-to-map-integer-to-a-specifi) which may be what you are after. – Alexei Levenkov Oct 08 '17 at 20:05

1 Answers1

0

There is nothing wrong with using a dictionary here as that is what you would most likely hand-roll if you were to make a custom solution. Also it supports nearly all LINQ methods you could think of.

Chibueze Opata
  • 9,856
  • 7
  • 42
  • 65
  • Program was killed after using 3.7GB RAM :-D [screenshot 0.7gb before death](http://ibb.co/nOSfJw) I will try it again tomorrow at better PC with huge RAM, but I don't think, it will help – SoptikHa Oct 08 '17 at 19:47
  • I just looked at your question again, you need to rethink what you're doing with your array/dictionary. Why do you need to hold so much data at once? Consider processing the data in chunks or on the fly, instead of waiting for the RAM to get exhausted. – Chibueze Opata Oct 08 '17 at 19:56
  • thanks, I'll look at things you've written as well as try to thing out another way to do it without using enormous collections – SoptikHa Oct 08 '17 at 20:18