WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Refine accessibility for custom controls

Refine accessibility for custom controls

AccessibilitySwiftUIVoiceOverCustom Controls
Watch original video

Highlight

Apple provides a complete set of accessibility APIs for custom controls, including adjustable traits, passthrough gestures, custom actions, and direct touch, so users who rely on assistive technologies can fully use complex interactive controls.

Core Content

Building a custom UI control is not hard. Making it usable for everyone is the hard part.

Imagine you built a stylish coffee control: users slide a finger up and down to adjust the amount of coffee. Sighted users understand it at a glance. The fill height represents the amount of coffee, and the gesture is dragging. The brain processes shape, position, and expected behavior in a fraction of a second.

But what about users who cannot see it? The track, handle, and position all disappear.

VoiceOver, Apple’s built-in screen reader, communicates that information through speech. A standard Slider control reads something like: “Brightness, 50%, adjustable. Swipe up or down with one finger to adjust the value.” The user knows what the control is, what the current value is, and how to operate it.

These are the four guiding principles for accessibility in custom controls:

  1. Clear purpose: what the control does
  2. Visible value: if it has a value, that value can be obtained
  3. Explicit actions: what actions the user can perform
  4. Timely feedback: what changes after an action

Many custom controls are designed only for visual interaction and ignore assistive technologies. The result is that users who rely on VoiceOver or Switch Control cannot use them at all.

Apple provides a complete set of APIs to solve this. From simple adjustable controls, to complex two-dimensional equalizers, to virtual pets with multiple gestures, each scenario has a corresponding tool.

Details

Make adjustable controls truly adjustable

(05:01)

CoffeeDispenserView is a custom slider that adjusts the amount of coffee by swiping up and down. But without accessibility support, VoiceOver only reads “button” and some unrelated information.

The first step is to mark it as an accessibility element and provide a clear label and value:

struct CoffeeDispenserView: View {
    @State var coffee: Double = 0.0
    var body: some View {
        CoffeeSlider(value: coffee)
            .accessibilityElement()
            .accessibilityLabel("Coffee Dispenser")
            .accessibilityValue("\(Int(coffee)) ounces")
            .accessibilityAddTraits(.adjustable)
            .accessibilityAdjustableAction { direction in
                switch direction {
                case .increment:
                    increaseCoffeeAmount()
                case .decrement:
                    decreaseCoffeeAmount()
                }
            }
    }
}

Key points:

  • .accessibilityElement() marks this as an independent accessibility element
  • .accessibilityLabel tells VoiceOver what it is
  • .accessibilityValue exposes the current value
  • The .adjustable trait indicates that this control can be adjusted
  • .accessibilityAdjustableAction defines what swiping up and down should do

Now VoiceOver can read: “Coffee Dispenser, 6 ounces, adjustable. Swipe up or down with one finger to adjust the value.”

Use passthrough gestures for precise control

(07:05)

Standard swiping adjusts by a fixed step, such as 1 ounce at a time. Some users need 0.5-ounce precision.

VoiceOver has a built-in capability called a passthrough gesture: double-tap and hold, and touch events are sent directly to the control.

struct CoffeeDispenserView: View {
    @State var coffee: Double = 0.0

    var body: some View {
        CoffeeSlider(value: coffee)
            .accessibilityActivationPoint(
                UnitPoint(x: 0.5, y: 1 - coffee)
            )
    }
}

Key points:

  • .accessibilityActivationPoint sets the gesture start point
  • x: 0.5 centers it horizontally, and y: 1 - coffee keeps the start point aligned with the current coffee amount
  • This gives the user’s finger room to move in any direction

Passthrough gestures also need feedback. Announcing every value change would be too noisy:

struct CoffeeDispenserView: View {
    @State var coffee: Double = 0.0

    var body: some View {
        CoffeeSlider(value: coffee)
            .onChange(of: coffee) { _, newValue in
                if sufficientTimeSinceLastAnnouncement() && valueHasChanged() {
                    cacheLastSpokenValue(newValue)
                    AccessibilityNotification
                        .Announcement(newValue)
                        .post()
                }
            }
    }
}

Key points:

  • Check whether more than 0.3 seconds have passed since the last announcement
  • Check whether the value actually changed
  • .post() sends the accessibility notification

After users double-tap and hold, they hear updates while moving their finger: “6.4 ounces… 8.4 ounces… 9.5 ounces”

Use custom actions for two-dimensional controls

(10:13)

The equalizer in iOS background sounds is a two-dimensional control that can move on both the frequency and amplitude axes. The standard .adjustable trait supports only one direction of increment and decrement, which is not enough.

This is where custom actions are needed:

struct EqualizerView: View {
    var body: some View {
        EqualizerPad()
            .accessibilityActions("Move Up") {
                increaseY(by: 10)
            }
            .accessibilityActions("Move Right") {
                increaseX(by: 10)
            }
            .accessibilityActions("Move Down") {
                decreaseY(by: 10)
            }
            .accessibilityActions("Move Left") {
                decreaseX(by: 10)
            }
    }
}

Key points:

  • .accessibilityActions can add multiple actions, each with a name and closure
  • The four directions control the X or Y axis separately
  • VoiceOver reads all available actions, and users swipe up or down to choose one, then double-tap to execute it

During operation, VoiceOver reads: “Move down. Move up. Move right. Frequency 0, amplitude 40.”

Use direct touch for multi-gesture controls

(12:47)

A virtual pet control supports three gestures: petting for a purr, tapping for a meow, and pinching for a hiss. Passthrough gestures are not enough because users may want to perform several actions repeatedly.

Direct Touch is the better choice:

struct VirtualCat: View {
    var cat: CatModel
    var body: some View {
        InteractiveCatSurface()
            .accessibilityLabel("Virtual Cat")
            .accessibilityValue(cat.currentReaction.description)
            .accessibilityDirectTouch([.requiresActivation])
    }
}

Key points:

  • .accessibilityDirectTouch marks a region as supporting direct touch
  • .requiresActivation requires a double-tap to activate it, avoiding accidental touches
  • Direct touch stays active until focus moves to another element
  • During direct touch, VoiceOver speaks instructions for how to operate the control

There is also a .silentOnTouch option for controls that provide their own audio feedback, preventing VoiceOver speech from overlapping with control audio.

Key Takeaways

Add accessibility support to a game controller simulator

What to build: Use custom actions for directional controls and A/B buttons so Switch Control users can play mobile games too.

Why it is worth building: Many mobile games depend on complex gesture operations, making them unusable for Switch Control users. Custom actions map each game action into a system-recognizable action item, expanding the audience.

How to start: Add .accessibilityActions to the game control, define independent action names and closures for each direction and button, and let VoiceOver users swipe up or down to choose an action and double-tap to execute it.

Adapt a two-dimensional equalizer for a music mixer

What to build: A two-dimensional equalizer can move across both frequency and amplitude axes. Use custom actions to support independent control of frequency and amplitude.

Why it is worth building: The standard .adjustable trait supports only one direction of increment and decrement. Two-dimensional controls need a more flexible action model. Custom actions let VoiceOver users precisely control each dimension.

How to start: Add multiple .accessibilityActions to the equalizer view, name them “Move Up,” “Move Down,” “Move Left,” and “Move Right,” and have each action adjust the corresponding coordinate.

Enable direct touch mode in a drawing app

What to build: Use Direct Touch with .silentOnTouch so VoiceOver users can draw directly with a finger and tool sounds are not interrupted by speech.

Why it is worth building: Drawing is visual art, but blind and low-vision users can also create through tactile and audio feedback. Direct touch lets finger input act directly on the canvas, and .silentOnTouch prevents VoiceOver speech from overlapping with drawing sound effects.

How to start: Add .accessibilityDirectTouch([.requiresActivation, .silentOnTouch]) to the canvas view. Use .requiresActivation to prevent accidental touches and .silentOnTouch to let control audio play exclusively.

Make fitness intensity controls accessible

What to build: Add the adjustable trait and passthrough gestures to a custom slider, supporting both coarse and fine adjustment.

Why it is worth building: Intensity controls in fitness apps usually depend on visual feedback such as fill height, which users who cannot see cannot perceive. The adjustable trait exposes the current value, while passthrough gestures support precise fine-tuning.

How to start: Add .accessibilityElement(), .accessibilityLabel, .accessibilityValue, and .accessibilityAddTraits(.adjustable). Use .accessibilityAdjustableAction for increment and decrement, and .accessibilityActivationPoint for passthrough gestures.

Adapt virtual pet interactions for direct touch

What to build: A virtual pet control supports several gestures, such as petting, tapping, and pinching. Use direct touch so VoiceOver users can experience the full interaction model.

Why it is worth building: The appeal of toy-like apps comes from varied interactions. Supporting only a single tap removes much of the experience. Direct touch keeps receiving touch events after activation and supports complex gesture combinations.

How to start: Add .accessibilityDirectTouch([.requiresActivation]) to the interactive region. During activation, VoiceOver speaks the available operations and users can directly perform the gestures.

Comments

GitHub Issues · utterances