Haskell - Turning multiple image-files into one video-file using the ffmpeg-light package

Question

Background
I wrote an application for image-processing which uses the ffmpeg-light package to fetch all the frames of a given video-file so that the program afterwards is able to apply grayscaling, as well as edge detection alogrithms to each of the frames.

Now I'm trying to put all of the frames back into a single video-file.

Used Libs
ffmpeg-light-0.12.0
JuicyPixels-3.2.8.3
...

What have I tried?
I have to be honest, I didn't really try anything because I'm kinda clueless where and how to start. I saw that there is a package called Command which allows running processes/commands using the command line. With that I could use ffmpeg (not ffmpeg-light) to create a video out of image-files which I would have to save to the hard drive first but that would be kinda hacky.

Within the documentation of ffmpeg-light on hackage (ffmpeg-light docu) I found the frameWriter function which sounds promising.

frameWriter :: EncodingParams -> FilePath -> IO (Maybe (AVPixelFormat, V2 CInt, Vector CUChar) -> IO ())

I guess FilePath would be the location where the video file gets stored but I can't really imagine how to apply the frames as EncodingParams to this function.

Others
I can access:

r, g, b, a as well asy. a values
image width / height / format

Question
Is there a way to achieve this using the ffmpeg-light package?

As the ffmpeg-light package lacks of documentation when it comes to conversion from images to video, I really would appreciate your help. (I do not expect a fully working solution.)

Code
The code that reads the frames:

-- Gets and returns all frames that a given video contains
getAllFrames :: String -> IO [(Double, DynamicImage)]
getAllFrames vidPath = do 
  result <- try (imageReaderTime $ File vidPath) :: IO (Either SomeException (IO (Maybe (Image PixelRGB8, Double)), IO()))
  case result of 
    Left ex -> do 
                 printStatus "Invalid video-path or invalid video-format detected." "Video" 
                 return []
    Right (getFrame, _) -> addNextFrame getFrame [] 

-- Adds up all available frames to a video.
addNextFrame :: IO (Maybe (Image PixelRGB8, Double)) -> [(Double, DynamicImage)] -> IO [(Double, DynamicImage)]
addNextFrame getFrame frames = do
  frame <- getFrame
  case frame of 
    Nothing -> do 
                 printStatus "No more frames found." "Video"
                 return frames
    _       -> do                             
                 newFrameData <- fmap ImageRGB8 . swap . fromJust <$> getFrame 
                 printStatus ("Frame: " ++ (show $ length frames) ++ " added.") "Video"
                 addNextFrame getFrame (frames ++ [newFrameData])

Where I am stuck / The code that should convert images to video:

-- Converts from several images to video
juicyToFFmpeg :: [Image PixelYA8] -> ?
juicyToFFmpeg imgs = undefined

You can do something like `frameWriter e f >>= \writeFrame -> mapM_ (writeFrame . Just) frames >> writeFrame Nothing` where `frames` is a list of frames in the appropriate format (a 3-tuple - `(AVPixelFormat, V2 CInt, Vector CUChar)`). Then you just need a function `Image PixelYA8 -> (AVPixelFormat, V2 CInt, Vector CUChar)` - can you write this function? — user2407038, Oct 23 '17 at 19:22
Thanks for that fast reply. While stepping through your code I finally understood how it works. — oRole, Oct 23 '17 at 22:26
If you have a new question, don't edit this old one. Just ask a new question. — user2407038, Oct 25 '17 at 21:04

HTNW · Accepted Answer · 2017-10-23T21:28:57.317

Disclaimer: not familiar with these libraries. Information gleaned from type signatures and documentation.

Use Codec.FFmpeg.Juicy:

Codec.FFmpeg.Juicy.imageWriter ::
  JuicyPixelFormat p =>
  EncodingParams ->
  FilePath ->
  IO (
    Maybe (Image p) -> 
    IO ()
  )

Define

instance JuicyPixelFormat PixelYA8 where
  juicyPixelFormat _  = _ -- in memory format of PixelYA8

juicyToFFmpeg :: [Image PixelYA8] -> FilePath -> IO ()
juicyToFFmpeg is fp = do writer <- imageWriter params fp
                         -- give Just image data to writer to append it
                         forM_ is (writer . Just)
                         writer Nothing -- finalize, or else you'll break it
   where params :: EncodingParams
         params = _ -- Figure out what fps, width, height, etc. you want (hardcode? parameters to juicyToFFmpeg? fold on is?)

Old Answer

juicyToFFmpeg :: [Image PixelYA8] -> FilePath -> IO ()
juicyToFFmpeg is fp = do writer <- frameWriter (findEncodingParams is) fp
                         forM_ is $ \img -> let form = imgForm img
                                                dims = imgDims img
                                                pixs = imgData img
                                            in writer $ Just (form, dims, pixs)
                         writer Nothing
   where findEncodingParams = _
         imgForm :: Image PixelYA8 -> AVPixelFormat
         imgForm = _ -- however your images are encoded
         imgDims :: Image PixelYA8 -> V2 Int
         imgDims = _ -- duh
         imgData :: Image PixelYA8 -> Vector CUInt
         imgData = _ -- Encode as a bunch of integers, following the value of imgForm

Thanks for your answer! It also helped me solving the problem. — oRole, Oct 23 '17 at 22:28

Haskell - Turning multiple image-files into one video-file using the ffmpeg-light package

1 Answers1

Old Answer