0

I'm working with C# in VS2022 Enviroment. I have a program that uses large array and pass these array to mkl dll. It seams debugging that C part receive a wrong data after 4GB block. While debugging C# side (looping the array) all values appear correct.

Any one have faced this kind of problem? consider that the application has this properties set: gcAllowVeryLargeObjects = "true" And is compiled for x64 platform.

It seams C# cannot alloc a block of memory more than 4G contiguously.

Or is it possible that the parameter declaration in C# is not properly configured?

This is how variables are declared in C#:

        int MAX_SIZE = 16385; //limit 16385
        Complex[,] A = new Complex[MAX_SIZE, MAX_SIZE];
        Complex[] b = new Complex[MAX_SIZE];

The arrays are populated than passed to the dll.

This is the dll declaration in C#:

 [DllImport("mkl_rt.2.dll", ExactSpelling = true, SetLastError = false, CallingConvention = CallingConvention.Cdecl)]
    internal static extern int LAPACKE_zgesvxx(
        int matrix_layout,
        char fact,
        char trans,
        Int64 n,
        Int64 nrhs,
        [In, Out] Complex* A,
        Int64 lda,
        [In, Out] Complex* af,
        Int64 ldaf,
        int[] ipiv,
        ref char equed,
        [In, Out] double* r,
        [In, Out] double* c,
        [In, Out] Complex* B,
        Int64 ldb,
        [In, Out] Complex* x,
        Int64 ldx,
        ref double rcond,
        ref double rpvgrw,
        [In, Out] double[] berr,
        int n_err_bnds,
        [In, Out] double[] err_bnds_norm,
        [In, Out] double[] err_bnds_comp,
        int nparams,
        [In, Out] double[] parameters
    );

Thank you

Tostone
  • 95
  • 1
  • 9
  • 3
    Here's a link to the Microsoft doc that may be useful. https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element – Dmyto Holota Jul 15 '22 at 07:37
  • I personally load limited chunks as list from the source file. If it is necessary you can store the rest items on a temp file. Or bTree. Or some other file structure. – redParrot Jul 15 '22 at 07:38
  • 2
    This might also be useful as well https://stackoverflow.com/questions/1391672/what-is-the-maximum-size-that-an-array-can-hold#:~:text=By%20default%2C%20the%20maximum%20size,total%20of%204%20billion%20elements. – Maytham Fahmi Jul 15 '22 at 07:42
  • duplicate: [Why doesn't C# support arrays larger than 2GB?](https://stackoverflow.com/q/46379694/995714), [How to Read A File More than 2 Gb Bytes into a Array in C#?](https://stackoverflow.com/q/45514854/995714), [The limitation on the size of .Net array](https://stackoverflow.com/q/2415434/995714). [BigArray, getting around the 2GB array size limit](https://learn.microsoft.com/en-us/archive/blogs/joshwil/bigarrayt-getting-around-the-2gb-array-size-limit) – phuclv Jul 15 '22 at 08:39
  • [NativeMemoryArray — A library that takes full advantage of the .NET 6 API to handle huge data of over 2GB](https://neuecc.medium.com/nativememoryarray-a-library-that-takes-full-advantage-of-the-net-31a2fa61b01) – phuclv Jul 15 '22 at 08:44
  • thanks for your suggestions, but I've already used gcAllowVeryLargeObjects = "true". Also, if I loop through the array, all values are correct. It seems like parameters are passed or handled in memory. – Tostone Jul 15 '22 at 13:21
  • @Tostone you need to edit and include that information in the question in order for it to be reopened. The question must be cleared – phuclv Jul 16 '22 at 09:54
  • 1
    I resolved the problem making many tries and I've discovered this: what I've understandood is that C# manages array 2D (n dimension?) contiguosly if the size is less than 4GB (because it works). Than, if the size is upper, this is not more valid, each dimension has a potential different allocated block (strange). In conclusion the 2D array pointer is not reliable. – Tostone Jul 19 '22 at 07:06

0 Answers0