I have installed Opencv 2.4.13.6 at my Ubuntu 16.04 OS.
I have ffmpeg and during Opencv installation I made WITH_FFMPEG ON.
My ffmpeg is working. 
If I type ffmpeg at command window, I have
ffmpeg version N-90982-gb995ec0 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609
  configuration: --prefix=/home/nyan/ffmpeg_build --enable-shared --extra-cflags=-I/home/nyan/ffmpeg_build/include --extra-ldflags=-L/home/nyan/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/nyan/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
  libavutil      56. 18.100 / 56. 18.100
  libavcodec     58. 19.100 / 58. 19.100
  libavformat    58. 13.101 / 58. 13.101
  libavdevice    58.  4.100 / 58.  4.100
  libavfilter     7. 21.100 /  7. 21.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
Then I have put ffmpeg paths to .bashrc as
export PATH=/home/bin${PATH:+:${PATH}}
export PATH=/home/ffmpeg_build${PATH:+:${PATH}}
export PATH=/home/ffmpeg_build/include${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/home/ffmpeg_build/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
In my Opencv libraries I have libopencv_video.so. So video input/output should be fine.
My following program gives me "can't read video". What could be the reason?
I tried VideoCapture cap(0); gives me same error. What is wrong?
#include <opencv2/opencv.hpp>
using namespace cv;
using namespace std;
int main(void){
    VideoCapture cap("IMG_5715.MOV"); // open the default camera
    if(!cap.isOpened())  // check if we succeeded
    {
        cout << "can't read video"<< endl;
        return -1;
    }
    while(1){ 
    Mat frame;
    // Capture frame-by-frame
    cap >> frame;
        imshow( "Frame", frame );
        waitKey(1);
    // If the frame is empty, break immediately
    if (frame.empty())
       break; 
    }
    cap.release();
    return 0;
}