WWDC Quick Look đź’“ By SwiftGGTeam
中文 EN 日本語
Support the Center Stage front camera in your iOS app

Support the Center Stage front camera in your iOS app

AVFoundationCameraCenter StageiOS 26
Watch original video

Highlight

The front camera in the iPhone 17 family uses a square sensor and a 95-degree field-of-view lens, enabling dynamic aspect ratio switching and smart framing so selfies and video calls automatically adapt to the number and position of people in the frame.

Core Content

Front-camera capture used to come with a few fixed frustrations.

For landscape or portrait photos, you had to rotate the phone. For group selfies, you had to ask someone else for help or stretch your arm farther. During video calls, a small movement could push someone out of frame, forcing the user to reposition manually.

The front camera on iPhone 17, iPhone Air, and iPhone 17 Pro solves these problems at the hardware level.

The sensor is square rather than a traditional 4:3 rectangle. That means you can choose any aspect ratio, landscape, portrait, or square, without rotating the phone. The lens reaches a 95-degree field of view, the widest ever on an iPhone front camera. Combined with face and gaze detection, the camera can automatically zoom and rotate the frame to keep everyone included.

Apple packages these capabilities into two APIs: Dynamic Aspect Ratio and Smart Framing Monitor. The first lets you switch frame orientation seamlessly, and the second automatically provides framing recommendations.

For video calls, Center Stage can adjust the frame in real time so participants stay centered. There is also a low-latency stabilization mode designed specifically for video call stability.

Details

Dynamic Aspect Ratio

(05:29)

Dynamic Aspect Ratio is the foundation for all manual and automatic framing. Starting in iOS 26, AVCaptureDevice adds a new dynamicAspectRatio property.

After you set this property, the camera crops the selected aspect ratio from the square sensor without rebuilding the capture session or interrupting the preview. The switch happens in real time.

The supported device type is .builtInUltraWideCamera, the front ultra-wide camera. It supports only square formats, with resolutions from 1280 to 4032. Five aspect ratios are available: 3:4, 4:3, 9:16, 16:9, and 1:1. The 4032 format supports only 3:4 and 4:3 because those two provide the highest photo resolution.

Here is the code for a “tap to rotate” button:

// Select the Center Stage front camera
let discoverySession = AVCaptureDevice.DiscoverySession(
    deviceTypes: [.builtInUltraWideCamera],
    mediaType: .video,
    position: .front
)
guard let camera = discoverySession.devices.first else { return }

// Find a format that supports the target aspect ratio
let desiredAspectRatio = AVCaptureDevice.AspectRatio.ratio4x3
let format = camera.formats.first { format in
    format.supportedDynamicAspectRatios.contains(desiredAspectRatio)
}

// Configure the camera
try? camera.lockForConfiguration()
camera.activeFormat = format
camera.activeVideoMinFrameDuration = CMTime(value: 1, timescale: 30)

// Switch the aspect ratio
let timestamp = camera.setDynamicAspectRatio(desiredAspectRatio)
camera.unlockForConfiguration()

Key points:

  • DiscoverySession looks only for a front camera of type .builtInUltraWideCamera
  • supportedDynamicAspectRatios returns the aspect ratios supported by the format
  • setDynamicAspectRatio returns the timestamp of the first frame where the change takes effect
  • The switch does not interrupt the preview, so users do not see a black frame

Smart Framing Monitor

(07:39)

Automatic zoom and rotation are driven by AVCaptureSmartFramingMonitor. Based on face and gaze detection, it periodically provides framing recommendations, including an aspect ratio and zoom factor. Your app can choose to accept or ignore them.

The monitor provides recommendations only in the 4032 photo format because it is designed specifically for photo capture.

Setup code:

// The camera was selected in the dynamic aspect ratio step
let format = camera.formats.first { format in
    format.supportedDynamicAspectRatios.contains(.ratio4x3) &&
    format.supportedFramings.contains(.smartFraming)
}

try? camera.lockForConfiguration()
camera.activeFormat = format
camera.unlockForConfiguration()

// Get and configure the smart framing monitor
let monitor = camera.smartFramingMonitor
monitor.enabledFramings = monitor.supportedFramings

// Observe framing recommendations
monitor.observe(\.recommendedFraming) { monitor, _ in
    guard let recommendation = monitor.recommendedFraming else { return }
    try? camera.lockForConfiguration()
    camera.setDynamicAspectRatio(recommendation.aspectRatio)
    camera.videoZoomFactor = recommendation.zoomFactor
    camera.unlockForConfiguration()
}

// Start monitoring
monitor.startMonitoring()

Key points:

  • smartFramingMonitor is an AVCaptureDevice property
  • enabledFramings controls which framing methods the monitor can recommend, and can be limited to a subset
  • Use KVO to observe the recommendedFraming property and receive live recommendations
  • A recommendation contains two values: aspectRatio and zoomFactor
  • Set the aspect ratio first, then the zoom factor, for a smoother preview transition
  • stopMonitoring() stops monitoring, and removeObserver() cancels observation

Sensor Orientation Compensation

(10:05)

The front camera sensors in earlier iPhone generations were mounted in landscape-left orientation. When taking a portrait photo, the image buffer carried an EXIF orientation tag telling the player to rotate it by 270 degrees.

The Center Stage front camera sensor in the iPhone 17 family is mounted in portrait orientation. If your code depends on the old rotation value, photos may appear sideways.

AVCapturePhotoOutput automatically handles orientation compensation by default. It physically rotates the photo, updates the EXIF metadata, and then delivers it to your app. The resulting photo, just like on older devices, is in landscape-left orientation, so you can continue using your existing rotation logic.

Compensation applies only to HEIC, JPEG, and uncompressed processed photos. It does not apply to Bayer RAW or Apple ProRAW.

iOS 26 adds a new cameraSensorOrientationCompensationEnabled property that can disable this behavior. Disabling it improves performance, but you need to ensure your rotation handling is correct.

Center Stage Video Calls

(14:44)

If your video calling app uses VoIP background mode, users can turn on Center Stage directly from the Video Effects menu in Control Center. System-level video effects such as Portrait, Studio Light, and gestures are enabled per process, and once active, they apply to all supported cameras in the app.

Apps that do not use VoIP mode can also manually enable Center Stage starting with the iPhone 17 family.

// Select a supported format
let format = camera.formats.first { format in
    format.supportedCenterStage == true
}

try? camera.lockForConfiguration()
camera.activeFormat = format
camera.centerStageControlMode = .cooperative  // Allow users to control it inside the app
camera.isCenterStageEnabled = true
camera.unlockForConfiguration()

Key points:

  • supportedCenterStage tells you whether the format supports Center Stage
  • centerStageControlMode has two modes: cooperative, where users can control it, and app, where only the app controls it
  • isCenterStageEnabled turns on Center Stage
  • Once enabled, the frame adjusts automatically to keep everyone centered

Low-Latency Stabilization

(15:47)

The Center Stage front camera also supports a real-time low-latency stabilization mode optimized specifically for video calls.

if let connection = videoOutput.connection(with: .video) {
    connection.preferredVideoStabilizationMode = .lowLatency
}

This mode is off by default. Once enabled, shake during calls is noticeably reduced and the image becomes more stable.

Core Ideas

1. Add a Smart Group Selfie Mode to a Selfie App

What to build: Use the Smart Framing Monitor to automatically detect the number of people in the frame, then automatically zoom and rotate so everyone appears in a good composition before the shutter is pressed.

Why it matters: In group selfies, someone is often cropped out or pushed to the edge, and users need to adjust position and distance repeatedly. The Smart Framing Monitor uses face and gaze detection to recommend framing, and after the app accepts it, the user only needs to press the shutter.

How to start: Get the smartFramingMonitor from AVCaptureDevice, enable smartFraming, observe changes to recommendedFraming with KVO, and automatically set dynamicAspectRatio and videoZoomFactor. Entry API: AVCaptureSmartFramingMonitor

2. Add Simultaneous Landscape and Portrait Support to a Live Streaming App

What to build: Use Dynamic Aspect Ratio so creators can switch between landscape and portrait without rotating the phone, adapting to recommended formats on different platforms.

Why it matters: Different live streaming platforms recommend different frame ratios, such as TikTok vertical 9:16 and Bilibili landscape 16:9. Previously, creators had to rotate the phone or reconfigure the setup. Dynamic Aspect Ratio switches in real time without interrupting the stream, so one button can adapt to multiple platforms.

How to start: Select the .builtInUltraWideCamera front camera, find a format that supports the target aspect ratio, and call setDynamicAspectRatio(_:) to switch. Entry API: camera.setDynamicAspectRatio(.ratio16x9)

3. Improve Multi-Person Join Experience in a Video Conferencing App

What to build: Enable Center Stage and low-latency stabilization so the frame automatically widens when someone enters, keeps everyone visible, and stays stable while the device is handheld.

Why it matters: People often move in and out during video meetings. When someone joins, the frame may not include everyone, and handheld shake can make the call uncomfortable to watch. Center Stage automatically tracks people, while low-latency stabilization reduces shake and improves call quality.

How to start: Select a format with supportedCenterStage, set centerStageControlMode = .cooperative, enable isCenterStageEnabled, and set preferredVideoStabilizationMode = .lowLatency on the video connection. Entry API: camera.isCenterStageEnabled = true plus connection.preferredVideoStabilizationMode = .lowLatency

4. Add Selfie Pose Guidance to a Social App

What to build: Combine Smart Framing Monitor recommendations with visual hints in the preview UI that tell users things like “move left a little” or “step back to include more.”

Why it matters: Most users do not know composition rules, so selfies often feel unbalanced. Translating the monitor’s framing recommendations into direct visual guidance can help users take better photos and improve the app’s camera experience.

How to start: Observe changes to smartFramingMonitor.recommendedFraming, compare the current framing with the recommendation, and overlay arrows or text prompts on the preview layer to guide the user. Entry API: monitor.observe(\.recommendedFraming)

5. Support 180-Degree Capture in a Camera App

What to build: Use the square sensor and 95-degree field of view to let users capture forward and backward directions in one shot, then stitch or crop the result into a panorama or comparison image.

Why it matters: The iPhone 17 square sensor and 95-degree ultra-wide lens are the widest ever on an iPhone front camera, covering a larger field of view than traditional front cameras in one capture. This enables new photo modes such as front-and-background compositions or ultra-wide selfies.

How to start: Use the 4032-resolution format on .builtInUltraWideCamera for the maximum field of view, then use the ultra-wide coverage after capture for image stitching or creative cropping. Entry API: filter the largest resolution from camera.formats plus .builtInUltraWideCamera

Comments

GitHub Issues · utterances