Dynamically changing video in OpenCV

Question

I have 3 video source connected to my PC and I want to show them on one screen. I initially started to put the video sources next to each other and that works fine, but I want to be able to enable/disable each video source at run time.

So I am want to use keyboard keys (r(right) and l(left)) to change what cameras are being shown at the moment.

I want to move the declaration of the following 3 variables to the outside of the while loop so I can access them in the if-cases and change them.

    cv::Mat3b combinedFrame(camRightSize.height, camMiddleSize.width + camRightSize.width);

    cv::Mat3b leftSideOfScreen(combinedFrame, Rect(0, 0, camMiddleSize.width, camMiddleSize.height));
    cameraMiddleFrameMirroredResize.copyTo(leftSideOfScreen);
    cv::Mat3b rightSideOfScreen(combinedFrame, Rect(camMiddleSize.width, 0, camRightSize.width, camRightSize.height));

Below is my whole code:

int main(int argc, char **argv) {

    int combinedScreenWidth = 1440;
    int combinedScreenHeight = 540;
    int rearCameraBiggerByThis = 200;
    int combinedScreenWidthHalv = combinedScreenWidth / 2;

    bool showRight = true;
    bool showLeft = false;

    //initialize and allocate memory to load the video stream from camera 

    cv::VideoCapture cameraRight(0);    // RIGHT
    cv::VideoCapture cameraMiddle(3);   // REAR
    cv::VideoCapture cameraLeft(3);     // LEFT

    if (!cameraRight.isOpened())    return 1;
    if (!cameraMiddle.isOpened())   return 1;
    if (!cameraLeft.isOpened())     return 1;

    cv::Mat3b cameraRightFrame;
    cv::Mat3b cameraMiddleFrame;
    cv::Mat3b cameraLeftFrame;

    cv::Mat3b cameraRightFrameMirrored;
    cv::Mat3b cameraMiddleFrameMirrored;
    cv::Mat3b cameraLeftFrameMirrored;

    Size camRightSize;
    Size camMiddleSize;
    Size camLeftSize;

    cv::Mat3b cameraRightFrameMirroredResize;
    cv::Mat3b cameraMiddleFrameMirroredResize;
    cv::Mat3b cameraLeftFrameMirroredResize;


    while (true) {
        // Grab and retrieve each frames of the video sequentially 
        cameraRight >> cameraRightFrame;
        cameraMiddle >> cameraMiddleFrame;
        cameraLeft >> cameraLeftFrame;
        // Mirror
        cv::flip(cameraRightFrame, cameraRightFrameMirrored, 1);
        cv::flip(cameraMiddleFrame, cameraMiddleFrameMirrored, 1);
        cv::flip(cameraLeftFrame, cameraMiddleFrameMirrored, 1);
        // Resize
        camRightSize = cameraRightFrame.size();
        camMiddleSize = cameraMiddleFrame.size();
        camLeftSize = cameraLeftFrame.size();

        resize(cameraMiddleFrameMirrored, cameraMiddleFrameMirroredResize, Size(combinedScreenWidthHalv + rearCameraBiggerByThis, combinedScreenHeight));
        resize(cameraRightFrameMirrored, cameraRightFrameMirroredResize, Size(combinedScreenWidthHalv - rearCameraBiggerByThis, combinedScreenHeight));

        // Compilation
        camRightSize = cameraRightFrameMirroredResize.size();
        camMiddleSize = cameraMiddleFrameMirroredResize.size();
        camLeftSize = cameraLeftFrameMirroredResize.size();

        if (showRight && showLeft) {    // LEFT + REAR + RIGHT

        } else if (showRight) {         // REAR + RIGHT

        } else if (showLeft) {          // LEFT + REAR

        } else {                        // REAR

        }

        cv::Mat3b combinedFrame(camRightSize.height, camMiddleSize.width + camRightSize.width);

        cv::Mat3b leftSideOfScreen(combinedFrame, Rect(0, 0, camMiddleSize.width, camMiddleSize.height));
        cameraMiddleFrameMirroredResize.copyTo(leftSideOfScreen);
        cv::Mat3b rightSideOfScreen(combinedFrame, Rect(camMiddleSize.width, 0, camRightSize.width, camRightSize.height));
        cameraRightFrameMirroredResize.copyTo(rightSideOfScreen);

        // declare windows
        cv:namedWindow("Combined", CV_WINDOW_NORMAL);
        cv::setWindowProperty("Combined", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
        cv::putText(combinedFrame, "REAR", cv::Point(500, 50), cv::FONT_HERSHEY_PLAIN, 2.0, cv::Scalar(255, 255, 255), 2 );

        cv::putText(combinedFrame, "RIGHT", cv::Point(950, 50), cv::FONT_HERSHEY_PLAIN, 2.0, cv::Scalar(255, 255, 255), 2 );


        cv::imshow("Combined", combinedFrame);          // 1440 x 540 Screen size
        //cv::imshow("Right Cam", cameraRightFrame);
        //cv::imshow("Middle Cam", cameraMiddleFrame);
        //cv::imshow("Left Cam", cameraLeftFrame);


        //wait for 40 milliseconds
        int c = cvWaitKey(1);

        //exit the loop if user press "Esc" key  (ASCII value of "Esc" is 27) 
        if (27 == char(c)) {
            break;
        }
        else if (114 == char(c)) {
            showRight = !showRight;
        }
        else if (108 == char(c)) {
            showLeft = !showLeft;
        }
    }

    return 0;
}

You're calling an overload that doesn't exist (that message is fairly clear about that). The closest I can see is [`Mat cv::Mat::operator() (const Rect & roi) const`](http://docs.opencv.org/3.1.0/d3/d63/classcv_1_1Mat.html#a07413f2e3e63a12185b8b218c24c7270). What's that statement supposed to be doing? — Dan Mašek, May 17 '17 at 11:52
Why `cv::Mat3b::Mat` instead of just `cv::Mat`? It's the same type, you just look it up from its derived class. | `int(camRightSize.width)` -- Since it's `cv::Size`, it's already an int. | Even if you like to type a lot (based on identifiers), more code means more potential for mistakes and more work to maintain it, so it's good to avoid redundancy. | Last thing - inconsistent namespace qualification. Just drop the `using namespace cv;`, it's a [bad idea anyway](http://stackoverflow.com/questions/1452721/why-is-using-namespace-std-considered-bad-practice). — Dan Mašek, May 17 '17 at 12:36

sansuiso · Answer 1 · 2017-05-17T12:39:15.450

0

According to your description, I think what you're wanting to write is:

combinedFrame(Rect(0, 0, camMiddleSize.width, camMiddleSize.height)).copyTo(leftSideOfScreen);

That is:

create a cv::Mat from another one (call to cv::Mat::copyTo(cv::Mat&))
the "source matrix" is obtained by extracting a rectangle from combinedFrame : combinedFrame(Rect(0, 0, camMiddleSize.width, camMiddleSize.height))

edited May 17 '17 at 12:39

answered May 17 '17 at 12:23

sansuiso

9,259
1
40
58

Uh, could you please point out the documentation (or implementation) of this particular operator overload? I'm not aware of its existence, nor is my compiler. | Even if it did exist, this would be rather awkward to read later, I'd much prefer a simple assignment, which makes it rather clear what happens. – Dan Mašek May 17 '17 at 12:30
Assignment will share the internal data. I guess in your case it's then better to copy or clone. Let me update the answer. – sansuiso May 17 '17 at 12:36
Sharing the data is kind of the point here, as far as I can tell. OP is grabbing a ROI from the full target frame, in order to copy the processed frame from camera into it (in the following statement). – Dan Mašek May 17 '17 at 12:38
Sharing the data should work, but I haven't tested with OpenCV 3.x that the variables describing the data row-by-row layout are still correctly updated. – sansuiso May 17 '17 at 12:41
i.e. `leftSideOfScreen = combinedFrame(Rect(0, 0, camMiddleSize.width, camMiddleSize.height));` – Dan Mašek May 17 '17 at 12:41
It was definitely doable with version 2.x. But to be tested with 3. – sansuiso May 17 '17 at 12:42
TBH, if this didn't still work in `3.x` (not that it doesn't), the author of that change would deserve to be tarred and feathered :D. Can you just imagine the amount of code such change in a core data structure would break for no good reason? I know there are some API changes, but this would be ridiculous. – Dan Mašek May 17 '17 at 12:48
What I've seen for sure is that the memory allocation strategy somehow changed. It used to generate image data that included padding so that start of rows gets aligned on specific boundaries, but now this padding has disappeared. It was a classical gotcha of OpenCV 2 (iterating over images as a single contiguous vector). But to be honest I haven't verified the other cases/constructors. – sansuiso May 17 '17 at 12:52
@sansuiso Hi, I don't understand the answer, maybe I explained it wrong, please see updated question. – theAlse May 18 '17 at 18:35
Sorry, I got lost by the edit... Do you want to have one big matrix form which you get the data to display? Or to write all the 3 camera images into the one big matrix? – sansuiso May 19 '17 at 19:32

Dynamically changing video in OpenCV

1 Answers1