Saving opencv Mat and MatOfKeyPoint [ ANDROID JAVA ]

Question

I am using SIFT by opencv in android studio JAVA language. I want to save the features of the image which are Mat and MatOfKeyPoint so that next for the same image I don't have to recalculate those features.

So far I have tried :

TinyDB

ISSUE : It gets the data corrupted when app is restarted. Below is the code I used to save for Mat, same I have applied for MatOfKeyPoint too

tinyDB.putObject("MAT_DATA", (Object) mat_descriptors);

for reading I am using this

Log.d("TINYDB_", "MAT : "+tinyDB.getObject("MAT_DATA", Mat.class)+"");

below are logs while I am reading object,

When app is not restarted after putObject

2022-08-15 11:15:35.166 5143-5143/com.programminghut.opencv D/TINYDB_: MAT : Mat [ 59868*128*CV_32FC1, isCont=true, isSubmat=false, nativeObj=0xb4000071f399bcb0, dataAddr=0xb4000070f2d8e000 ]

When app is restarted and putObject is not called

2022-08-15 11:16:09.405 5268-5268/com.programminghut.opencv D/TINYDB_: MAT : Mat [ -1*-1*CV_8UC1, isCont=true, isSubmat=false, nativeObj=0xb4000071f399bcb0, dataAddr=0x0 ]

Now I am looking either solution to the above tinyDB problem or If you think anything else could utilised to save Mat into local storage that would also be very helpful.

I am pretty sure that you can't just dump the mat object into a database. See this related question: [How to serialize/deserialize OpenCv Mat in Java?](https://stackoverflow.com/questions/53333710/how-to-serialize-deserialize-opencv-mat-in-java) — sorifiend, Aug 15 '22 at 05:57
@sorifiend THANKS! it works... but now it is out of memory problem it works fine for small Mat matrix; ERROR : Caused by: java.lang.OutOfMemoryError: Failed to allocate a 253488000 byte allocation with 25165824 free bytes and 163MB until OOM, target footprint 457461840, growth limit 603979776 — Pawandeep, Aug 15 '22 at 06:30
That is a different question, so let's close this one as solved. If you want to process a large file then you'll need to make sure that your device has enough free memory, and if not then you need to process the image in smaller chunks and save the individual parts separately. See [here for another solution that might work for you](https://stackoverflow.com/questions/32244851/androidjava-lang-outofmemoryerror-failed-to-allocate-a-23970828-byte-allocatio), although I still recommend checking the available memory and breaking the image into parts to process it. — sorifiend, Aug 16 '22 at 02:38
I would have closed it but the solution link you gave only works for MAT not for MatOfKeypoints, I found solution for matofkeypoint too I’ll add it here and then close. Also splitting image in pieces worked I did that already. THANKS! — Pawandeep, Aug 16 '22 at 08:59

score 1 · Accepted Answer · answered Aug 16 '22 at 09:10

Thanks to @sorifiend I was able to store Mat via tinyDB.

how-to-serialize-deserialize-opencv-mat-in-java :- It works for only for Mat

I did more googling and found a solution on stack overflow for MatOfKeyPoint too but can not find link to it.

So below is the class I made after combining both the solutions :


import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;

import org.opencv.core.KeyPoint;
import org.opencv.core.Mat;
import org.opencv.core.MatOfKeyPoint;
import org.opencv.core.Point;

class SerializeTheMat
{
    public static String keypointsToJson(MatOfKeyPoint mat){
        if(mat!=null && !mat.empty()){
            Gson gson = new Gson();

            JsonArray jsonArr = new JsonArray();

            KeyPoint[] array = mat.toArray();
            for(int i=0; i<array.length; i++){
                KeyPoint kp = array[i];

                JsonObject obj = new JsonObject();

                obj.addProperty("class_id", kp.class_id);
                obj.addProperty("x",        kp.pt.x);
                obj.addProperty("y",        kp.pt.y);
                obj.addProperty("size",     kp.size);
                obj.addProperty("angle",    kp.angle);
                obj.addProperty("octave",   kp.octave);
                obj.addProperty("response", kp.response);

                jsonArr.add(obj);
            }

            String json = gson.toJson(jsonArr);

            return json;
        }
        return "{}";
    }

    public static MatOfKeyPoint keypointsFromJson(String json){
        MatOfKeyPoint result = new MatOfKeyPoint();

        JsonParser parser = new JsonParser();
        JsonArray jsonArr = parser.parse(json).getAsJsonArray();

        int size = jsonArr.size();

        KeyPoint[] kpArray = new KeyPoint[size];

        for(int i=0; i<size; i++){
            KeyPoint kp = new KeyPoint();

            JsonObject obj = (JsonObject) jsonArr.get(i);

            Point point = new Point(
                    obj.get("x").getAsDouble(),
                    obj.get("y").getAsDouble()
            );

            kp.pt       = point;
            kp.class_id = obj.get("class_id").getAsInt();
            kp.size     =     obj.get("size").getAsFloat();
            kp.angle    =    obj.get("angle").getAsFloat();
            kp.octave   =   obj.get("octave").getAsInt();
            kp.response = obj.get("response").getAsFloat();

            kpArray[i] = kp;
        }

        result.fromArray(kpArray);

        return result;
    }







    //  Serialization/deserialization utility
    public static String SerializeFromMat(Mat mat)
    {
        SerializedMat serializedMat = new SerializedMat();
        serializedMat.setType(mat.type());
        serializedMat.setRows(mat.rows());
        serializedMat.setCols(mat.cols());

        if (serializedMat.getType()==0||
                serializedMat.getType()==8||
                serializedMat.getType()==16||
                serializedMat.getType()==24)
        {
            serializedMat.setBytes(new byte[(int)(mat.total()*mat.elemSize())]);
            mat.get(0,0,serializedMat.bytes);
        }
        else if (serializedMat.getType()==1||
                serializedMat.getType()==9||
                serializedMat.getType()==17||
                serializedMat.getType()==25)
        {
            serializedMat.setBytes(new byte[(int)(mat.total()*mat.elemSize())]);
            mat.get(0,0,serializedMat.bytes);
        }
        else if (serializedMat.getType()==2||
                serializedMat.getType()==10||
                serializedMat.getType()==18||
                serializedMat.getType()==26)
        {
            serializedMat.setShorts(new short[(int)(mat.total()*mat.elemSize())]);
            mat.get(0,0,serializedMat.shorts);
        }
        else if (serializedMat.getType()==3||
                serializedMat.getType()==11||
                serializedMat.getType()==19||
                serializedMat.getType()==27)
        {
            serializedMat.setShorts(new short[(int)(mat.total()*mat.elemSize())]);
            mat.get(0,0,serializedMat.shorts);
        }
        else if (serializedMat.getType()==4||
                serializedMat.getType()==12||
                serializedMat.getType()==20||
                serializedMat.getType()==28)
        {
            serializedMat.setInts(new int[(int)(mat.total()*mat.elemSize())]);
            mat.get(0,0,serializedMat.ints);
        }
        else if (serializedMat.getType()==5||
                serializedMat.getType()==13||
                serializedMat.getType()==21||
                serializedMat.getType()==29)
        {
            serializedMat.setFloats(new float[(int)(mat.total()*mat.elemSize())]);
            mat.get(0,0,serializedMat.floats);
        }
        else if (serializedMat.getType()==6||
                serializedMat.getType()==14||
                serializedMat.getType()==22||
                serializedMat.getType()==30)
        {
            serializedMat.setDoubles(new double[(int)(mat.total()*mat.elemSize())]);
            mat.get(0,0,serializedMat.doubles);
        }

//        return "";
        Gson gson = new Gson();
        return gson.toJson(serializedMat);
    }

    public static Mat DeserializeToMat(String json)
    {
        Gson gson = new Gson();
        SerializedMat serializedMat = gson.fromJson(json, SerializedMat.class);
        Mat mat = new Mat(serializedMat.getRows(),serializedMat.getCols(),serializedMat.getType());

        if (serializedMat.getType()==0||
                serializedMat.getType()==8||
                serializedMat.getType()==16||
                serializedMat.getType()==24)
        {
            mat.put(0,0,serializedMat.getBytes());
        }
        else if (serializedMat.getType()==1||
                serializedMat.getType()==9||
                serializedMat.getType()==17||
                serializedMat.getType()==25)
        {
            mat.put(0,0,serializedMat.getBytes());
        }
        else if (serializedMat.getType()==2||
                serializedMat.getType()==10||
                serializedMat.getType()==18||
                serializedMat.getType()==26)
        {
            mat.put(0,0,serializedMat.getShorts());
        }
        else if (serializedMat.getType()==3||
                serializedMat.getType()==11||
                serializedMat.getType()==19||
                serializedMat.getType()==27)
        {
            mat.put(0,0,serializedMat.getShorts());
        }
        else if (serializedMat.getType()==4||
                serializedMat.getType()==12||
                serializedMat.getType()==20||
                serializedMat.getType()==28)
        {
            mat.put(0,0,serializedMat.getInts());
        }
        else if (serializedMat.getType()==5||
                serializedMat.getType()==13||
                serializedMat.getType()==21||
                serializedMat.getType()==29)
        {
            mat.put(0,0,serializedMat.getFloats());
        }
        else if (serializedMat.getType()==6||
                serializedMat.getType()==14||
                serializedMat.getType()==22||
                serializedMat.getType()==30)
        {
            mat.put(0,0,serializedMat.getDoubles());
        }

        return mat;
    }

    private static class SerializedMat
    {
        byte[] bytes;
        short[] shorts;
        int[] ints;
        float[] floats;
        double[] doubles;

        int type;
        int rows;
        int cols;

        byte[] getBytes()
        {
            return bytes;
        }

        void setBytes(byte[] bytes)
        {
            this.bytes = bytes;
        }

        short[] getShorts()
        {
            return shorts;
        }

        void setShorts(short[] shorts)
        {
            this.shorts = shorts;
        }

        int[] getInts()
        {
            return ints;
        }

        void setInts(int[] ints)
        {
            this.ints = ints;
        }

        float[] getFloats()
        {
            return floats;
        }

        void setFloats(float[] floats)
        {
            this.floats = floats;
        }

        double[] getDoubles()
        {
            return doubles;
        }

        void setDoubles(double[] doubles)
        {
            this.doubles = doubles;
        }

        int getType()
        {
            return type;
        }

        void setType(int type)
        {
            this.type = type;
        }

        int getRows()
        {
            return rows;
        }

        void setRows(int rows)
        {
            this.rows = rows;
        }

        int getCols()
        {
            return cols;
        }

        void setCols(int cols)
        {
            this.cols = cols;
        }

        SerializedMat()
        {
        }
    }
}

And you can store using tinyDB like below

String kp_to_json=SerializeTheMat.keypointsToJson(kp_ref);
String descriptors =  SerializeTheMat.SerializeFromMat(data);

tinyDB.putString("DESC", descriptors);
tinyDB.putString("KP", kp_to_json);

And get it back like follows:

data = SerializeTheMat.DeserializeToMat(tinyDB.getString("DESC"));

kp_ref = SerializeTheMat.keypointsFromJson(tinyDB.getString("KP"));

For storing them locally you can write JSON file from those string and read those JSON files whenever needed.

THANKS

Saving opencv Mat and MatOfKeyPoint [ ANDROID JAVA ]

1 Answers1