8

I'm try create application where I can record video from different cameras on device during recording. For example. User press button "start record" from front camera, after 5 second recording user press button "Switch Camera" and application change video source from front to back camera and recording continue. For camera swithcing I use next code:

NSError *error;
AVCaptureDeviceInput *newVideoInput;
AVCaptureDevicePosition currentCameraPosition = [[videoInput device] position];

if (currentCameraPosition == AVCaptureDevicePositionBack)
{
    currentCameraPosition = AVCaptureDevicePositionFront;
}
else
{
    currentCameraPosition = AVCaptureDevicePositionBack;
}

AVCaptureDevice *backFacingCamera = nil;
NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
for (AVCaptureDevice *device in devices) 
{
    if ([device position] == currentCameraPosition)
    {
        backFacingCamera = device;
    }
}
newVideoInput = [[AVCaptureDeviceInput alloc] initWithDevice:backFacingCamera error:&error];

if (newVideoInput != nil)
{
    [_captureSession beginConfiguration];

    [_captureSession removeInput:videoInput];
    if ([_captureSession canAddInput:newVideoInput])
    {
        [_captureSession addInput:newVideoInput];
        videoInput = newVideoInput;
    }
    else
    {
        [_captureSession addInput:videoInput];
    }
    //captureSession.sessionPreset = oriPreset;
    [_captureSession commitConfiguration];
}

_inputCamera = backFacingCamera;

After this appication change video sourse and continue writing, but... audio/video out of sync... Can anybody hehp me with this problem?

Thank you.

kroumvud
  • 358
  • 6
  • 19

3 Answers3

3

I had the same problem, tried a lot if things and came up with an easy and convenient solution which works without any glitch.

enter image description here

The problem is, as @daij-djan pointed out, that switching a session input takes a little time and adds a few black frames to the output and then continue sending frames as if it has never stop. As far as I know it is impossible to know how much frames are impacted. The session timecode is not impacted by this delay so we can't use it to ignore some video frames.

enter image description here

Instead of having one session with multiple camera inputs, you can have one session per camera input (+ one for audio if you need it) and one video output. Then you only need to switch this output between the sessions.

enter image description here

As a result you will have no desynchronization and no ugly hack to do. The memory impact is very limited in my tests and I didn't noticed any performance impact. My theory is that a session isn't active until it has an output attached.

Switching outputs can be done like this:

fromSession.beginConfiguration()
fromSession.removeOutput(videoOutput)
fromSession.commitConfiguration()

toSession.beginConfiguration()
if toSession.canAddOutput(videoOutput) { toSession.addOutput(videoOutput) }
toSession.commitConfiguration()
Macistador
  • 854
  • 12
  • 23
  • Could you not simply capture the time when the switch camera begins, stop the session, make the change, start the session again and capture the time when frames are captured again to determine the "lag" and just adjust the timestamp for all frames thereafter to account for the lag? You may also need to drop any frames which come through with timestamps during the lag. – JCutting8 Mar 12 '19 at 09:27
  • How to switch output btw Capture Sessions without stopping video recording? – George Heints Apr 21 '20 at 12:32
  • 1
    @GeorgeHeints I edited my response to add a sample code – Macistador Apr 21 '20 at 16:11
  • Really appreciate this answer! I’ve been working on a library with someone about a year ago (HybridCamera on github) and the pod works great. The recording is super smooth. I would just like to implement this feature, too. Do you have a project where you implemented it this way? If so, would you mind sending me some more code of how you did this so I could look into it and rewrite some of the files of the pod. I’m relatively new to AVFoundation and could need some more context. This solution sounds like the best one I’ve heard yet, though. Would love to hear from you! – Moritz Jun 19 '20 at 22:54
  • I tried making it work but I somehow can't pull it off. I always run into issues with the preview layer of the preview view. I would really appreciate your help if you have some time to spare. – Moritz Jun 20 '20 at 23:19
  • When we should call a toSession.startRunning()? At the same time as fromSession.startRunning()? – Rodion Kuskov Jul 13 '20 at 15:44
  • @Macistador How you deal with preview layers? You would have to keep switching up the session that causes some black frames right? Also you said one session per audio but you can only have output on one session at a time right? –  Jul 08 '21 at 12:23
  • @Moritz Did you ever find the solution? – Awais Fayyaz Dec 22 '22 at 17:35
1

you need to stop recording, switch and start it again. the camera switch isnt instant AFAIK

cant you record into n files and later stich them together?

try using individual AVMutableComposition tracks and then setting a mutablecomposition for audio and one for video. (see Merging two m4v Movie Files Using AVMutableComposition - Videos Will Not Merge)

Community
  • 1
  • 1
Daij-Djan
  • 49,552
  • 17
  • 113
  • 135
  • Thank you for your reply. And if I stop recording and change the camera, is it possible to somehow continue recording in the same file – kroumvud Dec 19 '12 at 13:19
  • sorry, I dont know but... cant you just record to N file and stich them together later on? – Daij-Djan Dec 19 '12 at 13:34
  • you know, if every time stop recording and then start a new one, that each stop old and start a new recording, application will freezing for a short period of time... and this is no good (( – kroumvud Dec 19 '12 at 13:49
  • Apps like snapChat do this without affecting the audio track. Perhaps 2 separate recorders? – Andres Canella Nov 04 '16 at 16:24
  • This is pretty slow in performance. So if you need the video right away after recording, this is probably not the right option. – slow Jun 26 '20 at 17:37
0

when switching camera, you need to pause the buffer writing in the delegate method:

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection

you can checkout the open source project PBJVision

Tunaki
  • 132,869
  • 46
  • 340
  • 423
Ruobin Wang
  • 358
  • 3
  • 9
  • Do you actually have this working on your end? PBJVision has the same [issue](https://github.com/piemonte/PBJVision/issues/328) that this question is referring to. – Andres Canella Nov 04 '16 at 16:22