Is there front facing camera support with ARKit?

ios swift arkit arscnview arcamera
Bharrath picture Bharrath · Oct 12, 2017 · Viewed 7.4k times · Source

How can we access Front Facing Camera Images with ARCamera or ARSCNView and is it possible to record ARSCNView just like Camera Recording?

Answer

rickster picture rickster · Oct 12, 2017

Regarding the front-facing camera: in short, no.

ARKit offers two basic kinds of AR experience:

  • World Tracking (ARWorldTrackingConfiguration), using the back-facing camera, where a user looks "through" the device at an augmented view of the world around them. (There's also AROrientationTrackingConfiguration, which is a reduced quality version of world tracking, so it still uses only the back-facing camera.)

  • Face Tracking (ARFaceTrackingConfiguration), supported only with the front-facing TrueDepth camera on iPhone X, where the user sees an augmented view of theirself in the front-facing camera view. (As @TawaNicolas notes, Apple has sample code here... which, until iPhone X actually becomes available, you can read but not run.)

In addition to the hardware requirement, face tracking and world tracking are mostly orthogonal feature sets. So even though there's a way to use the front facing camera (on iPhone X only), it doesn't give you an experience equivalent to what you get with the back facing camera in ARKit.

Regarding video recording in the AR experience: you can use ReplayKit in an ARKit app same as in any other app.

If you want to record just the camera feed, there isn't a high level API for that, but in theory you might have some success feeding the pixel buffers you get in each ARFrame to AVAssetWriter.

Related questions

ARKit Place a SCNNode facing the camera

I'm using ARKit to display 3D objects. I managed to place the nodes in the real world in front of the user (aka the camera). But I don't manage to make them to face the camera when I drop them. …

Read More
Unable to activate constraint with anchors

So, I'm trying to create a sceneView programatically class ViewController: UIViewController, ARSCNViewDelegate { var sceneView: ARSCNView = ARSCNView() let configuration = ARWorldTrackingConfiguration() override func viewDidLoad() { super.viewDidLoad() self.sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints, ARSCNDebugOptions.showWorldOrigin] self.configuration.planeDetection = .horizontal self.sceneView.session.run(configuration) …

Read More
How to detect vertical planes in ARKit?

How is it possible to implement a vertical plane detection (i.e. for walls)? let configuration = ARWorldTrackingSessionConfiguration() configuration.planeDetection = .horizontal //TODO

Read More