WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Enhance your app’s audio recording capabilities

Enhance your app’s audio recording capabilities

Audio & VideoSpatial ComputingPhotos & Camera
Watch original video

Highlight

iOS 26 brings four things to recording apps: in-app microphone switching, high-quality Bluetooth recording over AirPods, spatial audio capture via AVAssetWriter, and an Audio Mix at playback time that balances voice against ambience.

Core content

Recording apps have long been stuck on an awkward step: the user plugs in a new microphone, then has to leave the app, open Settings, pick the device, and come back. AirPods are just as awkward — the Bluetooth link is tuned for phone calls, so the recorded voice sounds flat while background noise pours in. Spatial audio is the other end of the story: iOS 18 only let AVCaptureMovieFileOutput capture it, so an audio-only app such as Voice Memos had no API to record spatial audio at all.

iOS 26 answers all four pain points at once. AVInputPickerInteraction moves input device selection inside the app, with a live level meter and microphone modes; the system remembers the choice. bluetoothHighQualityRecording opens a Bluetooth link tuned for content creation on AirPods, balancing voice against ambience. AVAssetWriter can now write a spec-compliant .qta file containing one AAC stereo track, one APAC spatial audio track, plus a metadata track. On playback, the Cinematic framework exposes the Audio Mix — the same capability the Photos app uses when editing a spatial audio video — letting the user switch between Cinematic, Studio, and In-Frame styles, plus an intensity slider that controls the foreground/background ratio.

Details

Input device selection (02:10). AVInputPickerInteraction is an NSObject subclass. Attach it to any UIView to bring up the input picker UI. The audio session must be configured first, otherwise the list will be wrong.

import AVKit

class AppViewController {

    // Configure AudioSession

    // AVInputPickerInteraction is a NSObject subclass that presents an input picker
    let inputPickerInteraction = AVInputPickerInteraction()
    inputPickerInteraction.delegate = self

    // connect the PickerInteraction to a UI element for displaying the picker
    @IBOutlet weak var selectMicButton: UIButton!
    self.selectMicButton.addInteraction(self.inputPickerInteraction)

    // button press callback: present input picker UI
    @IBAction func handleSelectMicButton(_ sender: UIButton) {
        inputPickerInteraction.present()
    }
}

Key points:

  • AVInputPickerInteraction() creates the picker instance; delegate must be the hosting view controller.
  • addInteraction attaches it to a real UI control (a UIButton here).
  • present() brings up the picker on tap, showing the available devices, a live level meter, and the microphone modes view.
  • Configure the audio session before calling, so the picker shows the correct set of devices.

High-quality recording over AirPods (03:57). This is a Bluetooth link tuned for content creation, sitting between a LAV mic’s voice and ambience balance. AVAudioSession opts in via a category option; AVCaptureSession opts in via a capture session property.

// AVAudioSession clients opt-in - session category option
AVAudioSessionCategoryOptions.bluetoothHighQualityRecording

// AVCaptureSession clients opt-in - captureSession property
session.configuresApplicationAudioSessionForBluetoothHighQualityRecording = true

Key points:

  • bluetoothHighQualityRecording is a new option on AVAudioSessionCategoryOptions, coexisting with the existing allowBluetoothHFP. When both are on, high quality is the default and HFP is the fallback.
  • configuresApplicationAudioSessionForBluetoothHighQualityRecording = true lets the capture session take over audio session configuration, saving manual setup.
  • Once enabled, the system input menu shows a “high-quality AirPods” entry in the device list.

Spatial audio capture (from 05:11). iOS 26 opens spatial audio writing on the AVAssetWriter path. A valid spatial audio file contains two audio tracks (AAC stereo + APAC FOA) and at least one metadata track. FOA (First Order Ambisonics) has 4 channels: 1 omnidirectional component plus 3 dipole components pointing along X/Y/Z.

Key APIs in practice:

  • AVCaptureDeviceInput.multichannelAudioMode = .firstOrderAmbisonics switches the microphone array into FOA mode.
  • An AVCaptureSession in FOA mode can carry up to two AudioDataOutputs — one outputting FOA, one outputting stereo — corresponding to the two audio tracks.
  • The new spatialAudioChannelLayoutTag property decides each ADO’s output format (Stereo or FOA).
  • AVCaptureSpatialAudioMetadataSampleGenerator takes the FOA buffers, emits a timestamped metadata sample at the end of recording, and writes it to a third AVAssetWriterInput.
  • iOS 26 also lets AVCaptureMovieFileOutput and AVAudioDataOutput run together, so you can write the file and draw a live waveform or apply effects at the same time.

Audio Mix on playback (13:26). Audio Mix uses an intensity (0–1) and a style enum to control the voice-to-ambience ratio.

import Cinematic

// Audio Mix parameters (consider using UI elements to change these values)
var intensity: Float32 = 0.5 // values between 0.0 and 1.0
var style = CNSpatialAudioRenderingStyle.cinematic

// Initializes an instance of CNAssetAudioInfo for an AVAsset asynchronously
let audioInfo = try await CNAssetSpatialAudioInfo(asset: myAVAsset)

// Returns an AVAudioMix with effect intensity and rendering style.
let newAudioMix: AVAudioMix = audioInfo.audioMix(effectIntensity: intensity,
                                                 renderingStyle: style)

// Set the new AVAudioMix on your AVPlayerItem
myAVPlayerItem.audioMix = newAudioMix

Key points:

  • CNAssetSpatialAudioInfo(asset:) reads the spatial audio metadata from the asset asynchronously.
  • audioMix(effectIntensity:renderingStyle:) returns an AVAudioMix, which can be set directly on AVPlayerItem.audioMix for playback.
  • CNSpatialAudioRenderingStyle includes the same Cinematic / Studio / In-Frame styles as the Photos app, plus 6 extra modes (such as a voice-only mono stem and an ambience-only FOA stem).
  • Drive intensity from a UI slider in real time to match the slider experience in the demo.

Low level: pulling remix metadata from the input file (16:45). When going through the AUAudioMix AudioUnit path, you need to feed the mix parameters embedded at recording time into the AU.

// Get Spatial Audio remix metadata from input AVAsset

let audioInfo = try await CNAssetSpatialAudioInfo(asset: myAVAsset)

// extract the remix metadata. Set on AUAudioMix with AudioUnitSetProperty()
let remixMetadata = audioInfo.spatialAudioMixMetadata as CFData

Key points:

  • spatialAudioMixMetadata returns the mix parameters written into the file at recording time, typed as CFData.
  • Set it on AUAudioMix via AudioUnitSetProperty(), so the AU works with the tuning from the recording session; without it, the result drifts from the recorded intent.
  • This path suits apps that need precise control over the Audio Mix inside a custom render graph; for normal playback, the AVPlayer path above is simpler.

Takeaways

  • What to do: add a “choose microphone” button to your recording or podcast app, calling AVInputPickerInteraction.

    • Why it’s worth doing: the most common drop-off is “I plugged in a mic but couldn’t switch to it.” Saving one trip out to system Settings can save one recording.
    • How to start: factor out the audio session setup, then attach the picker to an existing button on your recording toolbar, following the four-line pattern from 02:10.

  • What to do: turn on bluetoothHighQualityRecording for every recording flow that uses AirPods.

    • Why it’s worth doing: the code change is one option or one bool, but the audio jumps from “phone-call grade” to “content-creation grade” — a clear win for interviews, podcasts, and vlog narration.
    • How to start: audit every place you set AVAudioSessionCategoryOptions, and add the high-quality option alongside HFP. On the AVCaptureSession path, set configuresApplicationAudioSessionForBluetoothHighQualityRecording = true and let the system fall back to HFP.

  • What to do: upgrade audio-only apps in the Voice Memos mold to record spatial audio.

    • Why it’s worth doing: iOS 26 is the first release where AVAssetWriter can produce a compliant .qta. Whoever ships first reaps the downstream benefits — Audio Mix, AirPods head tracking, and so on.
    • How to start: scaffold from the sample project “Capturing Spatial Audio in your iOS app” — one FOA AudioDataOutput + one stereo AudioDataOutput + one metadata generator + three AVAssetWriterInputs, with the alternate track group set up per TN3177.

  • What to do: add a “voice / ambience” slider to an existing spatial audio player.

    • Why it’s worth doing: it lets the user rescue noisy material themselves, so fewer recordings get thrown away.
    • How to start: use CNAssetSpatialAudioInfo + audioMix(effectIntensity:renderingStyle:), expose three styles plus a slider that drives intensity directly. See 13:26.

  • What to do: merge recording and live waveform / effects into a single AVCaptureSession.

    • Why it’s worth doing: from iOS 26 on, MovieFileOutput and AudioDataOutput can coexist, removing the sync pain of running two sessions and cutting CPU in half.
    • How to start: collapse the two pipelines, let AudioDataOutput handle UI feedback only, and let MovieFileOutput focus on writing to disk.

Comments

GitHub Issues · utterances