WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Record, replay, and review: UI automation with Xcode

Record, replay, and review: UI automation with Xcode

SwiftDeveloper Tools
Watch original video

Highlight

Xcode can now record your tap operations as XCUITest code in one click, replay them across multiple languages, devices, and system states on Xcode Cloud, and pinpoint failures with video-enabled Test Reports.

Core Content

Many developers do not write UI tests. The reason is simple: hand-writing XCUIElementQuery chains is too verbose, and cross-language, cross-device verification relies on manual tapping. A simple “add to favorites” flow needs to run separately in English, Arabic, and German, then again on iPhone and iPad—the cost is so high that no one wants to do it.

Apple’s Xcode team splits UI automation into three stages: Record, Replay, and Review. In the recording stage, you just tap—Xcode translates each tap, swipe, and keyboard input into Swift code (12:00). In the replay stage, Test Plan maps the same test to multiple locale and device configurations, runnable locally or on Xcode Cloud. In the review stage, Test Report provides full video playback—failure moments get diamond markers, tapping elements jumps directly to code via View Source (03:00).

The underlying logic is worth noting: UI automation is built on the accessibility framework (05:02). accessibilityIdentifier is the “stable key” you give automation—it does not localize, is not read by screen readers, and exists solely for test code to locate elements. Get accessibility right first, and UI tests become stable.

Details

1. Add accessibility identifiers to views

SwiftUI uses the .accessibilityIdentifier modifier, combined with the data model’s id to make each cell unique (07:52):

import SwiftUI

struct LandmarkDetailView: View {
  let landmark: Landmark
  var body: some View {
    VStack {
      Image(landmark.backgroundImageName)
        .accessibilityIdentifier("LandmarkImage-\(landmark.id)")

      Text(landmark.description)
        .accessibilityIdentifier("LandmarkDescription-\(landmark.id)")
    }
  }
}

Key points:

  • LandmarkImage-\(landmark.id): appending the data id to the identifier ensures each cell in the list is unique.
  • Do not write identifiers like "Great Barrier Reef"—user-visible strings change with localization.
  • The three principles for identifiers: unique (unique within the app), descriptive (clearly named), static (does not change with content).

UIKit uses property assignment (08:19):

import UIKit

struct LandmarksListViewController: UIViewController {
  let landmarks: [Landmark] = [landmarkGreatBarrier, landmarkCairo]

  override func viewDidLoad() {
    super.viewDidLoad()

    for landmark in landmarks {
      let button = UIButton(type: .custom)
      setupButtonView()

      button.accessibilityIdentifier = "LandmarkButton-\(landmark.id)"

      view.addSubview(button)
    }
  }
}

Key points:

  • accessibilityIdentifier is a built-in UIView property; for controls that are accessibility elements by default (buttons, text, images), just assign it.
  • It will not be read by VoiceOver, nor exposed to users—it is a “private” field for automation.
  • Want to cut corners? Xcode’s coding assistant supports natural language commands like “Add accessibility identifiers to the relevant parts of this view” and will automatically use the model’s id field to build identifiers (09:05).

2. Rewrite queries immediately after recording

The recorder provides a dropdown for each line of code, listing multiple query methods. Max gave three selection principles (13:54):

// Counterexample: using localized strings
XCUIApplication().staticTexts["Max's Australian Adventure"]

// Good: using accessibility identifier
XCUIApplication().staticTexts["Collection-1"]

Key points:

  • Localized strings change in Arabic and German; identifiers do not.
  • staticTexts[...] is syntactic sugar for querying by identifier, matching the .accessibilityIdentifier you wrote in SwiftUI.

Second: keep queries as short as possible (14:09):

// Counterexample: chain too long
XCUIApplication().scrollViews.staticTexts["Collection-1"]

// Good: direct location
XCUIApplication().staticTexts["Collection-1"]

Key points:

  • An extra .scrollViews layer hard-codes “must be inside a ScrollView”—if you later change to List or Form, tests break.
  • When identifiers are unique across the app, skipping intermediate containers is safe.

Third: use generic queries for dynamic content (14:21):

// Counterexample: hard-coded content
XCUIApplication().staticTexts["Max's Australian Adventure"]

// Good: take first match
XCUIApplication().staticTexts.firstMatch

Key points:

  • firstMatch fits scenarios where “there should be text here, but the exact content varies.”

3. Add assertions, set device state, pass launch arguments

Recording is just the first step—add XCTAssert and waitForExistence (15:49):

import XCTest

class LandmarksUITests: XCTestCase {

  func testGreatBarrierAddedToFavorites() {
    let app = XCUIApplication()
    app.launch()
    app.cells["Landmark-186"].tap()
    XCTAssertTrue(
      app.staticTexts["Landmark-186"].waitForExistence(timeout: 10.0)),
      "Great Barrier exists"
    )

    let favoriteButton = app.buttons["Favorite"]
    favoriteButton.tap()
    XCTAssertTrue(
      favoriteButton.wait(for: \.value, toEqual: true, timeout: 10.0),
      "Great Barrier is a favorite"
    )
  }
}

Key points:

  • waitForExistence(timeout:) waits for elements to appear, avoiding flaky tests from animation or network delays.
  • wait(for: \.value, toEqual: true, timeout:) is a new keypath-based wait API that waits for any property to reach an expected value—much cleaner than sleep loops.
  • The favorite button’s value becomes true when selected—this is the state field exposed by the accessibility framework.

Device state can be set once in setUp (16:36):

import XCTest
import CoreLocation

class LandmarksUITests: XCTestCase {

  override func setUp() {
    continueAfterFailure = false

    XCUIDevice.shared.orientation = .portrait
    XCUIDevice.shared.appearance = .light

    let simulatedLocation = CLLocation(latitude: 28.3114, longitude: -81.5535)
    XCUIDevice.shared.location = XCUILocation(location: simulatedLocation)
  }
}

Key points:

  • XCUIDevice.shared.appearance switches light/dark mode, saving manual Dark Mode toggling.
  • XCUIDevice.shared.location = XCUILocation(location:) injects a simulated location into Simulator—useful for maps and nearby search apps.
  • continueAfterFailure = false ensures tests stop immediately when assertions fail, rather than continuing to tap in a broken state.

Launch arguments and environment variables control initial data (16:54):

import XCTest

class LandmarksUITests: XCTestCase {

  func testLaunchWithDefaultCollection() {
    let app = XCUIApplication()
    app.launchArguments = ["ClearFavoritesOnLaunch"]
    app.launchEnvironment = ["DefaultCollectionName": "Australia 🐨 🐠"]
    app.launch()

    app.tabBars.buttons["Collections"].tap()
    XCTAssertTrue(app.buttons["Australia 🐨 🐠"].waitForExistence(timeout: 10.0))
  }
}

Key points:

  • launchArguments is read inside the app via CommandLine.arguments—can trigger test branches like “clear favorites.”
  • launchEnvironment is read via ProcessInfo.processInfo.environment—better for passing initial values.
  • This mechanism makes each test independently repeatable, not dependent on state left by previous tests.

4. Use URL schemes to jump directly to pages, run accessibility audits

URL schemes can skip navigation layers and land directly on the target page (17:04):

import XCTest

class LandmarksUITests: XCTestCase {

  func testOpenGreatBarrier() {
    let app = XCUIApplication()
    let customURL = URL(string: "landmarks://great-barrier")!
    app.open(customURL)

    XCTAssertTrue(app.wait(for: .runningForeground, timeout: 10.0))
    XCTAssertTrue(app.staticTexts["Great Barrier Reef"].waitForExistence(timeout: 10.0))
  }
}

Key points:

  • app.open(customURL) lets your app handle the URL itself—saves layers of tapping when testing deep pages.
  • To simulate system-level jumps like “tapping a link in browser,” use XCUIDevice.shared.system.open(customURL) (17:12).
  • wait(for: .runningForeground, timeout:) waits for the app to actually enter foreground before asserting.

One final gem: run accessibility audits in one line (17:13):

import XCTest

class LandmarksUITests: XCTestCase {

  func testPerformAccessibilityAudit() {
    let app = XCUIApplication()
    try app.performAccessibilityAudit()
  }
}

Key points:

  • performAccessibilityAudit() automatically scans the current screen for contrast, hit target, missing label issues, and fails the test when found.
  • Add this test to CI to automatically catch accessibility regressions on every PR.

5. Test Plan multi-locale replay and Test Report

Test Plan multiplies the same test code across multiple configurations: each locale, device appearance, and screen orientation is a configuration set. Xcode Cloud distributes these configurations to cloud machines running in parallel, covering iPhone, iPad and other devices, and languages like English, Arabic, Greek, German. Test Report aggregates videos from each run: interaction points are overlaid with dots, failure moments get diamond markers, tapping elements shows code suggestions, and View Source jumps back to the test source file (03:22).

Key Takeaways

  • What to do: Add accessibilityIdentifier to all views in critical flows before writing UI tests.

    • Why: identifiers are the stable primary key for UI automation. Without them, recorded code is full of localized strings that break when run in multiple languages.
    • How: Pick 1 core flow (registration, checkout, favorites), use Xcode coding assistant with one prompt to batch add; follow naming convention <Type>-<id>.

  • What to do: Configure Test Plan with three locales by default—local language + one RTL (Arabic) + one long-string language (German).

    • Why: RTL validates layout mirroring, long strings validate truncation and wrapping. These two cover 80% of internationalization bugs.
    • How: In .xctestplan Configurations, add three configurations and change Localization for each; run locally first, then push to Xcode Cloud.

  • What to do: Add app.performAccessibilityAudit() to CI-required tests.

    • Why: One line of code catches accessibility regressions on every PR—far cheaper than fixing after user complaints post-launch.
    • How: Create a new UI test method, run audit once per core page; reuse setup from existing UI tests.

  • What to do: Immediately change default queries to identifier or firstMatch after recording; remove intermediate layers.

    • Why: Recorded queries are overly specific and break in batches after UI refactoring. Fix them while fresh to reduce maintenance cost by an order of magnitude.
    • How: Click dropdown on each line, prefer identifier; if missing, add to source code and re-record.

  • What to do: Change video and screenshot retention strategy from “failures only” to “keep all.”

    • Why: Passed run videos can directly serve as product demos, new hire onboarding, and marketing material—at nearly zero cost.
    • How: In Test Plan Configurations panel, change “UI Test Screenshots” and “UI Test Recording” to Always.

Comments

GitHub Issues ¡ utterances