Why is pointPolygonTest() method of OpenCV4Android returning -1 for every pixel?

Question

In the following code, I have carried out the following steps:

Loaded an image from sdcard.

Converted it to HSV format.
Used inRange function to mask out the red color.

Used findContours to find the contours.
Find the largest contour from those contours.
Created an ROI around the largest contour using boundingRect and submat functions.

Converted this ROI Mat to HSV format.

Iterated through the ROI Mat, and check for each pixel if it lies within the largest contour. I used the method pointPolygonTest to find this out, but it returns -1 for every pixel, as can be seen from the Log.i output I have pasted here. The question is why? How can I correct this.

private Scalar detectColoredBlob() {
    rgbaFrame = Highgui.imread("/mnt/sdcard/DCIM/rgbaMat4Mask.bmp");

    Mat hsvImage = new Mat();
    Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_BGR2HSV);
    Highgui.imwrite("/mnt/sdcard/DCIM/hsvImage.bmp", hsvImage);// check

    Mat maskedImage = new Mat();
    Core.inRange(hsvImage, new Scalar(0, 100, 100), new Scalar(10, 255, 255), maskedImage);
    Highgui.imwrite("/mnt/sdcard/DCIM/maskedImage.bmp", maskedImage);// check

    List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
    Imgproc.findContours(maskedImage, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);

    // \/ We will use only the largest contour. Other contours (any other possible blobs of this color range) will be ignored.
    MatOfPoint largestContour = contours.get(0);
    double largestContourArea = Imgproc.contourArea(largestContour);
    for (int i = 1; i < contours.size(); ++i) {// NB Notice the prefix increment.
        MatOfPoint currentContour = contours.get(i);
        double currentContourArea = Imgproc.contourArea(currentContour);
        if (currentContourArea > largestContourArea) {
            largestContourArea = currentContourArea;
            largestContour = currentContour;
        }
    }
    MatOfPoint2f largestContour2f = new MatOfPoint2f(largestContour.toArray());// Required on Line 289. See http://stackoverflow.com/questions/11273588/how-to-convert-matofpoint-to-matofpoint2f-in-opencv-java-api

    Rect detectedBlobRoi = Imgproc.boundingRect(largestContour);
    Mat detectedBlobRgba = rgbaFrame.submat(detectedBlobRoi);
    Highgui.imwrite("/mnt/sdcard/DCIM/detectedBlobRgba.bmp", detectedBlobRgba);// check

    Mat detectedBlobHsv = new Mat();
    Imgproc.cvtColor(detectedBlobRgba, detectedBlobHsv, Imgproc.COLOR_BGR2HSV);
    Highgui.imwrite("/mnt/sdcard/DCIM/roiHsv.bmp", detectedBlobHsv);// check

    for (int firstCoordinate = 0; firstCoordinate < detectedBlobHsv.rows(); firstCoordinate++) {
        for (int secondCoordinate = 0; secondCoordinate < detectedBlobHsv.cols(); secondCoordinate++) {
            Log.i(TAG, "HAPPY " + Arrays.toString(detectedBlobHsv.get(firstCoordinate, secondCoordinate)));
            if (Imgproc.pointPolygonTest(largestContour2f, new Point(firstCoordinate, secondCoordinate), false) == -1) {
                Log.i(TAG, "HAPPY ....................... OUTSIDE");
            }
        }
    }
    Highgui.imwrite("/mnt/sdcard/DCIM/processedcontoured.bmp", detectedBlobHsv);// check

EDIT:

I am doing this because I need to compute the average HSV color of pixels lying inside the contour (i.e. the average HSV color of the biggest red colored blob). If I computed the average color of the ROI detectedBlobHsv by the normal formula, I would do something like

Scalar averageHsvColor= new Scalar(256);
Scalar sumHsvOfPixels = new Scalar(256);
sumHsvOfPixels = Core.sumElems(detectedBlobHsv); 
int numOfPixels = detectedBlobHsv.width() * detectedBlobHsv.height(); 
for (int channel=0; channel<sumHsvOfPixels.val.length; channel++) { 
  averageHsvColor = sumHsvOfPixels.val[channel]/numOfPixels; 
}

So somebody here on SO (probably you?) had suggested me a way to exclude pixels outside my contour a while back. I'd implement that like:

//Giving pixels outside contour of interest an HSV value of `double[]{0,0,0}`, so that they don't affect the computation of `sumHsvOfPixels` while computing average, 
//and while keeping track of the number of pixels removed from computation this way, so we can subtract that number from the `$numOfPixels` during computation of average.
int pixelsRemoved = 0;
for (int row=0; row<detectedBlobHsv.rows(); row++) {
  for (int col=0; col<detectedBlobHsv.cols(); col++) {
    if (Imgproc.pointPolygonTest(largestContour2f, new Point(row, col), false) == -1) {
      detectedBlobHsv.put(row, col, new double[]{0,0,0});
      pixelsRemoved++;
    }
  }
}

Then compute the average like

Scalar averageHsvColor= new Scalar(256);
Scalar sumHsvOfPixels = new Scalar(256);
sumHsvOfPixels = Core.sumElems(detectedBlobHsv); //This will now exclude pixels outside the contour
int numOfPixels = (  detectedBlobHsv.width()*detectedBlobHsv.height()  )-pixelsRemoved; 
for (int channel=0; channel<sumHsvOfPixels.val.length; channel++) { 
  averageHsvColor = sumHsvOfPixels.val[channel]/numOfPixels; 
}

EDIT 1:

Towards the end of the following method, I have created the mask with a list of MatOfPoints which contains the largest contour only. When I wrote it to SDCard, I got

I don't know where I messed up!

private Scalar detectColoredBlob() {
        //Highgui.imwrite("/mnt/sdcard/DCIM/rgbaFrame.jpg", rgbaFrame);// check
        rgbaFrame = Highgui.imread("/mnt/sdcard/DCIM/rgbaMat4Mask.bmp");


        //GIVING A UNIFORM VALUE OF 255 TO THE V CHANNEL OF EACH PIXEL (255 IS THE MAXIMUM VALUE OF V ALLOWED - Simulating a maximum light condition) 
        for (int firstCoordinate = 0; firstCoordinate < rgbaFrame.rows(); firstCoordinate++) {
            for (int secondCoordinate = 0; secondCoordinate < rgbaFrame.cols(); secondCoordinate++) {
                double[] pixelChannels = rgbaFrame.get(firstCoordinate, secondCoordinate); 
                pixelChannels[2] = 255;
                rgbaFrame.put(firstCoordinate, secondCoordinate, pixelChannels);
            }
        }


        Mat hsvImage = new Mat();
        Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_BGR2HSV);
        Highgui.imwrite("/mnt/sdcard/DCIM/hsvImage.bmp", hsvImage);// check


        Mat maskedImage = new Mat();
        Core.inRange(hsvImage, new Scalar(0, 100, 100), new Scalar(10, 255, 255), maskedImage);
        Highgui.imwrite("/mnt/sdcard/DCIM/maskedImage.bmp", maskedImage);// check


        // Mat dilatedMat = new Mat();
        // Imgproc.dilate(maskedImage, dilatedMat, new Mat());
        // Highgui.imwrite("/mnt/sdcard/DCIM/dilatedMat.jpg", dilatedMat);// check


        List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
        Imgproc.findContours(maskedImage, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
        //FINDING THE BIGGEST CONTOUR
        // \/ We will use only the largest contour. Other contours (any other possible blobs of this color range) will be ignored.
        MatOfPoint largestContour = contours.get(0);
        double largestContourArea = Imgproc.contourArea(largestContour);
        for (int i = 1; i < contours.size(); ++i) {// NB Notice the prefix increment.
            MatOfPoint currentContour = contours.get(i);
            double currentContourArea = Imgproc.contourArea(currentContour);
            if (currentContourArea > largestContourArea) {
                largestContourArea = currentContourArea;
                largestContour = currentContour;
            }
        }


        Rect detectedBlobRoi = Imgproc.boundingRect(largestContour);
        Mat detectedBlobRgba = rgbaFrame.submat(detectedBlobRoi);
        Highgui.imwrite("/mnt/sdcard/DCIM/detectedBlobRgba.bmp", detectedBlobRgba);// check


        Mat detectedBlobHsv = new Mat();
        Imgproc.cvtColor(detectedBlobRgba, detectedBlobHsv, Imgproc.COLOR_BGR2HSV);
        Highgui.imwrite("/mnt/sdcard/DCIM/roiHsv.bmp", detectedBlobHsv);// check

        List<MatOfPoint> largestContourList = new ArrayList<>();
        largestContourList.add(largestContour);

        Mat roiWithMask = new Mat(detectedBlobHsv.rows(), detectedBlobHsv.cols(), CvType.CV_8UC3);
        roiWithMask.setTo(new Scalar(0,0,0));
        Imgproc.drawContours(roiWithMask, largestContourList, 0, new Scalar(0, 255, 255), -1);//TODO Using -1 instead of CV_FILLED.
        Highgui.imwrite("/mnt/sdcard/DCIM/roiWithMask.bmp", roiWithMask);// check


        // CALCULATING THE AVERAGE COLOR OF THE DETECTED BLOB
        // STEP 1:
        double [] averageHsvColor = new double[]{0,0,0};
        int numOfPixels = 0;
        for (int firstCoordinate = 0; firstCoordinate < detectedBlobHsv.rows(); ++firstCoordinate) {
            for (int secondCoordinate = 0; secondCoordinate < detectedBlobHsv.cols(); ++secondCoordinate) {

                double hue = roiWithMask.get(firstCoordinate, secondCoordinate)[0];
                double saturation = roiWithMask.get(firstCoordinate, secondCoordinate)[1];
                double value = roiWithMask.get(firstCoordinate, secondCoordinate)[2];

                averageHsvColor[0] += hue;
                averageHsvColor[1] += saturation;
                averageHsvColor[2] += value;

                numOfPixels++;
            }
        }
        averageHsvColor[0] /= numOfPixels;
        averageHsvColor[1] /= numOfPixels;
        averageHsvColor[1] /= numOfPixels;



        return new Scalar(averageHsvColor);
    }

EDIT 2:

I corrected my 3 channel mask and made a single channel mask

Mat roiMask = new Mat(rgbaFrame.rows(), rgbaFrame.cols(), CvType.CV_8UC1);
        roiMask.setTo(new Scalar(0));
        Imgproc.drawContours(roiMask, largestContourList, 0, new Scalar(255), -1);

and this resulted in the correct roiMask:

Then, before the comment // CALCULATING THE AVERAGE COLOR OF THE DETECTED BLOB, I added:

Mat newImageWithRoi = new Mat(rgbaFrame.rows(), rgbaFrame.cols(), CvType.CV_8UC3);
newImageWithRoi.setTo(new Scalar(0, 0, 0));
rgbaFrame.copyTo(newImageWithRoi, roiMask);
Highgui.imwrite("/mnt/sdcard/DCIM/newImageWithRoi.bmp", newImageWithRoi);//check

This resulted in:

Now again I don't know how to proceed. :s

Why you need to do that? Once you have the contour, or the mask, you already now which points are inside. — Miki, Jan 13 '16 at 20:50
@Miki Please see the edit in the question, I stated it there. — Solace, Jan 14 '16 at 04:34
I _hardly_ reccomended to use `pointPolygonTest` :D. See answer... — Miki, Jan 14 '16 at 16:43

score 1 · Accepted Answer · answered Jan 14 '16 at 16:43

1

You don't need to use pointPolygonTest, because you already have the mask.

You can simply sum up the values that lies on the mask. Something along the lines of (not able to test this):

// Initialize at 0!!!
Scalar averageHsvColor= new Scalar(0,0,0);

int numOfPixels = 0;

for(int r=0; r<detectedBlobHsv.height(); ++r)
{
    for(int c=0; c<detectedBlobHsv.width(); ++c)
    {
        if( /* value of mask(r,c) > 0 */) 
        {
            int H = // get H value of pixel at (r, c)
            int S = // get S value of pixel at (r, c)
            int V = // get V value of pixel at (r, c)

            // Sum values
            averageHsvColor[0] += H;
            averageHsvColor[1] += S;
            averageHsvColor[2] += V; 

            // Increment number of pixels inside mask
            numOfPixels ++;
        }
    }
}

// Compute average
averageHsvColor[0] /= numOfPixels ;
averageHsvColor[1] /= numOfPixels ;
averageHsvColor[2] /= numOfPixels ;

answered Jan 14 '16 at 16:43

Miki

40,887
13
123
202

_"because you already have the mask"_, and _"`if( /* value of mask(r,c) > 0 */)...`"_ - Which mask are we talking about? Do you mean I should create a mask from `detectedBlobHsv` by doing `Imgproc.drawContours(anEmptyMatForMask, aListHavingLargestContour, 0, new Scalar(255), -1);`? – Solace Jan 14 '16 at 17:39
In that case, I can't have something like `if (anEmptyMatForMask.get(r,c) > 0)` because `anEmptyMatForMask.get(r,c) ` would return a double, so it can't be compared with `0` :s – Solace Jan 14 '16 at 17:42
1

1) Yes! 2) Why you can't compare a double with `> 0`? – Miki Jan 14 '16 at 18:03
Sorry my bad! `anEmptyMatForMask.get(r,c)` would return a `double []`, i.e. a double array, whcih can't be compared to int. Anyways I tried it without this `if`, posting the results in an edit int the question. – Solace Jan 14 '16 at 18:07
1

`anEmptyMatForMask.get(r,c)[0]` should get the correct value – Miki Jan 14 '16 at 18:09
But that would check only the hue channel, may be I should add this check for saturation and value separately, like `anEmptyMatForMask.get(r,c)[1]` and `anEmptyMatForMask.get(r,c)[2]`. – Solace Jan 14 '16 at 18:20
1

The mask has a single channel only – Miki Jan 14 '16 at 18:20
I had it that way in the start: `Mat roiWithMask = new Mat(detectedBlobHsv.rows(), detectedBlobHsv.cols(), CvType.CV_8UC1); roiWithMask.setTo(new Scalar(0)); Imgproc.drawContours(roiWithMask, largestContourList, 0, new Scalar(255), -1);` but that gives me ***"java.lang.ArrayIndexOutOfBoundsException: length=1; index=1"*** at `double saturation = roiWithMask.get(firstCoordinate, secondCoordinate)[1];`, because for single channel Mat, the double[] returned by `roiWithMask.get(firstCoordinate, secondCoordinate)` will have only a single element. :s – Solace Jan 14 '16 at 18:27
I added another edit in the question. The main problem is that masks are single-channel Mats, whereas to compute average, we need all three channels of the original image. I think Masks are not gonna do, I'll have to go with `pointPolygonTest` (though I'd really prefer doing without that - have already wasted more than a day on that), don't you think so? – Solace Jan 14 '16 at 19:13
1

With the mask (single channel) you only select if a pixel position `(r,c)` should be used for computing the average or not. Then you compute it on your 3 channel hsv image at position `(r,c)` – Miki Jan 14 '16 at 19:16

Why is pointPolygonTest() method of OpenCV4Android returning -1 for every pixel?

1 Answers1