Run multiple AVCaptureSessions or add multiple inputs

jfischer picture jfischer · Jun 17, 2012 · Viewed 8.1k times · Source

I want to display the stream of the front and the back facing camera of an iPad2 in two UIViews next to each other. To stream the image of one device I use the following code

AVCaptureDeviceInput *captureInputFront = [AVCaptureDeviceInput deviceInputWithDevice:[AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo] error:nil];

AVCaptureSession *session = [[AVCaptureSession alloc] init];
session addInput:captureInputFront];
session setSessionPreset:AVCaptureSessionPresetMedium];
session startRunning];

AVCaptureVideoPreviewLayer *prevLayer = [AVCaptureVideoPreviewLayer layerWithSession:session];
prevLayer.frame = self.view.frame;
[self.view.layer addSublayer:prevLayer];

which works fine for either camera. To display the stream in parallel I tried to create another session, but as soon as the 2nd session is established the first freezes.

Then I tried to add two AVCaptureDeviceInput to the session but seems like at most one input is supported at the moment.

Any helpful ideas how to stream from both cameras?

Answer

Kevin Tonon picture Kevin Tonon · May 12, 2015

It is possible to get CMSampleBufferRefs from multiple video devices on MacOS X. You have to setup the AVCaptureConnection objects manually. For example, assuming you have these objects:

AVCaptureSession *session;
AVCaptureInput *videoInput1;
AVCaptureInput *videoInput2;
AVCaptureVideoDataOutput *videoOutput1;
AVCaptureVideoDataOutput *videoOutput2;

Do NOT add the outputs like this:

[session addOutput:videoOutput1];
[session addOutput:videoOutput2];

Instead, add them and tell the session not to make any connections:

[session addOutputWithNoConnections:videoOutput1];
[session addOutputWithNoConnections:videoOutput2];

Then for each input/output pair make the connection from the input's video port to the output manually:

for (AVCaptureInputPort *port in [videoInput1 ports]) {
    if ([[port mediaType] isEqualToString:AVMediaTypeVideo]) {
        AVCaptureConnection* cxn = [AVCaptureConnection
            connectionWithInputPorts:[NSArray arrayWithObject:port]
            output:videoOutput1
        ];
        if ([session canAddConnection:cxn]) {
            [session addConnection:cxn];
        }
        break;
    }
}

Finally, make sure to set sample buffer delegates for both outputs:

[videoOutput1 setSampleBufferDelegate:self queue:someDispatchQueue];
[videoOutput2 setSampleBufferDelegate:self queue:someDispatchQueue];

and now you should be able to process frames from both devices:

- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
       fromConnection:(AVCaptureConnection *)connection
{
    if (captureOutput == videoOutput1)
    {
        // handle frames from first device
    }
    else if (captureOutput == videoOutput2)
    {
        // handle frames from second device
    }
}

See also the AVVideoWall sample project for an example of combining live previews from multiple video devices.