Take SwiftUI to the next dimension

Highlight

SwiftUI uses Volume containers, Model3D APIs, RealityView attachments and 3D gestures on visionOS to allow developers to build bounded 3D experiences using familiar declarative syntax to display and manipulate 3D content without taking over the entire space.

Core Content

Volume: fixed size 3D container

Window will dynamically scale based on distance, Volume is different - it maintains a fixed size, no matter how far away from you, a 1 meter wide Volume will always be 1 meter wide. Volume is aligned horizontally to support viewing from any angle.

Creating a Volume simply requires applying a new window style on the WindowGroup:

@main
struct WorldApp: App {
    var body: some Scene {
        WindowGroup {
            ContentView()
        }
        .windowStyle(.volumetric)
    }
}

Key points:

• .windowStyle(.volumetric) turns a normal window into a Volume container
• There is no glass background in Volume, 3D content is placed directly into the scene
• Content can be displayed together with the control panel

(02:14)

Model3D: AsyncImage for 3D resources

Model3D is a SwiftUI view for loading 3D resources such as USDZ. The design concept is similar to AsyncImage, and the loading life cycle is managed through phase:

struct MoonView: View {
    var body: some View {
        Model3D(named: "Moon") { phase in
            switch phase {
            case .empty:
                ProgressView()
            case .failure(let error):
                Text(error.localizedDescription)
            case .success(let model):
                model
                    .resizable()
                    .scaledToFit()
            }
        }
    }
}

Key points:

• Model3D(named:) loads the USDZ file with the specified name from the resource package
• phase has three states: empty (loading), failure (failure), success (success)
• .resizable() tells the layout system that the model can be resized based on available space
• .scaledToFit() keeps the model scaled within the available space

(03:35)

3D layout and depth alignment

When multiple 3D models are placed side by side, back face aligned is used by default, that is, the back faces of all models are flush:

struct CelestialObjectView: View {
    let name: String
    let size: CGFloat

    var body: some View {
        Model3D(named: name) { phase in
            switch phase {
            case .success(let model):
                model
                    .resizable()
                    .scaledToFit()
                    .frame(width: size, height: size, depth: size)
                    .frame(depth: size, alignment: .front)
            default:
                EmptyView()
            }
        }
    }
}

Key points:

• .frame(width:height:depth:) sets the three-dimensional dimensions of the 3D view
• .frame(depth:alignment:) controls the alignment in the depth direction
• .front aligns the front of the model, .back (default) aligns the back
• Differences in alignment are apparent when viewed from an oblique side

(05:43)

3D animation effect

SwiftUI provides Rotation3DEffect for three-dimensional rotation:

TimelineView(.animation) { timeline in
    CelestialObjectView(name: object.name, size: object.size)
        .rotation3DEffect(
            Angle(degrees: timeline.date.timeIntervalSinceReferenceDate * 10),
            axis: .y
        )
}

Key points:

• TimelineView(.animation) provides a continuously updated time source
• .rotation3DEffect(axis:) rotates the view around the specified axis
• Rotate around the y-axis here to simulate the rotation effect of celestial bodies

(07:09)

Detailed Content

RealityView Attachments: Embed SwiftUI views into 3D scenes

RealityView’s attachments API lets you bind SwiftUI views to RealityKit entities. These views are alive - they can respond to state changes, run animations, and handle gestures.

RealityView { content, attachments in
    // Load the Earth model
    let earth = try! Entity.load(named: "Earth.usdz")
    content.add(earth)

    // Create label attachments for each location
    for place in favoritePlaces {
        if let label = attachments.entity(for: place.id) {
            content.add(label)
            // Position the label on the Earth surface
            label.lookAt(earth, from: place.location, relativeTo: earth)
        }
    }
} attachments: {
    ForEach(favoritePlaces) { place in
        Text(place.name)
            .glassBackgroundEffect()
            .tag(place.id)
    }
}

Key points:

• The attachments parameter provides access to the markup view
• The view is marked by .tag(), and the corresponding entity is obtained by attachments.entity(for:) in RealityView
• .glassBackgroundEffect() ensures labels are readable in any context
• lookAt(from:relativeTo:) positions the attachment towards the specified location
• Attachment view supports full SwiftUI functionality including status updates and gestures

(08:39)

Configure input reception for entities

To make entities in RealityView respond to gestures, you need to add two components:

RealityView { content in
    let earth = try! Entity.load(named: "Earth.usdz")

    // Add an input target component to receive input
    earth.components.set(InputTargetComponent())

    // Add a collision component to define the interaction area shape
    earth.components.set(
        CollisionComponent(shapes: [.generateSphere(radius: 1.0)])
    )

    content.add(earth)
}

Key points:

• InputTargetComponent() marks the entity and its descendants as being able to receive input
• CollisionComponent defines the geometry of the interaction area
• Here a sphere collider is used to match the shape of the earth model
• The accuracy of the collision body affects the accuracy of gesture positioning

(11:32)

Space click gesture

SpatialTapGesture provides the 3D click position:

RealityView { content in
    // ... Configure the Earth entity
}
.gesture(
    SpatialTapGesture()
        .targetedToEntity(earthEntity)
        .onEnded { value in
            // Get the 3D tap position in SwiftUI local coordinate space, in points
            let location3D = value.location3D

            // Convert to scene coordinate space
            let sceneLocation = value.convert(location3D, from: .local, to: earthEntity)

            // Add a new location above the tap position
            let newPlace = FavoritePlace(
                name: "New Discovery",
                location: sceneLocation * 1.05 // Slightly above the surface
            )
            favoritePlaces.append(newPlace)
        }
)

Key points:

• .targetedToEntity() limits the gesture to the specified entity, and the gesture fails when clicking on other areas.
• value.location3D Get the 3D coordinates of the click
• value.convert(...) converts positions between coordinate spaces
• The coordinate unit is points, which needs to be converted to scene space for use.

(12:15)

Combine 3D operation gestures

Combine drag, zoom and rotate into a complete gesture:

var manipulationGesture: some Gesture<AffineTransform3D> {
    DragGesture()
        .simultaneously(with: MagnifyGesture())
        .simultaneously(with: RotateGesture3D())
        .map { gesture in
            let (translation, scale, rotation) = gesture.components()
            return AffineTransform3D(
                scale: scale,
                rotation: rotation,
                translation: translation
            )
        }
}

// Helper method to extract gesture components
extension SimultaneousGesture<
    SimultaneousGesture<DragGesture, MagnifyGesture>,
    RotateGesture3D
>.Value {
    func components() -> (Vector3D, Size3D, Rotation3D) {
        let translation = self.first?.first?.translation3D ?? .zero
        let magnification = self.first?.second?.magnification ?? 1
        let size = Size3D(
            width: magnification,
            height: magnification,
            depth: magnification
        )
        let rotation = self.second?.rotation ?? .identity
        return (translation, size, rotation)
    }
}

Use gestures to control entity transformations:

@State private var isDragging = false

RealityView { content, attachments in
    // ... Configure content
} update: { content, attachments in
    if let satellite = content.entities.first(where: { $0.name == "satellite" }) {
        satellite
            .gesture(
                manipulationGesture
                    .updating($isDragging) { _, state, _ in
                        state = true
                    }
                    .onChanged { value in
                        satellite.transform = Transform(value)
                    }
            )
    }
}

Key points:

• The translation3D attribute of DragGesture provides the 3D translation amount
• RotateGesture3D measures unconstrained 3D rotation
• simultaneously(with:) Combine multiple gestures into simultaneously recognized gesture groups
• AffineTransform3D encapsulates translation, rotation and scaling of 3D transformations
• .updating tracks the instantaneous state of gestures and automatically resets when gestures fail
• Gestures support direct hand interaction, indirect pinch, trackpad and accessibility

(17:26)

Core Takeaways

• What to build: Create a 3D product display application that allows users to view products from any angle

  • Why it’s worth doing: Volume containers allow 3D content to coexist with other applications in Shared Space. Model3D loads the USDZ model asynchronously and automatically adapts the layout with .resizable() and .scaledToFit()
  • How to start: Use .windowStyle(.volumetric) to create a Volume, use Model3D to load the product model, and add Rotation3DEffect to allow the model to automatically rotate for display.

• What to build: Develop an annotated 3D map or anatomical diagram application

  • Why it’s worth doing: RealityView attachments allow SwiftUI views to be attached to precise locations on a 3D model. Labels can respond to status updates and support glass backgrounds to ensure readability
  • How to start: Create a Text label in RealityView’s attachments closure, mark it with .tag(), and use attachments.entity(for:) in RealityView closure to obtain and position it on the model surface

• What to build: Build a 3D design tool that supports gesture operations

  • Why it’s worth doing: Combination gestures (drag + zoom + rotate) allow users to manipulate 3D objects with natural hand movements. targetedToEntity ensures that the gesture accurately affects the target object
  • How to start: Define DragGesture.simultaneously(with: MagnifyGesture).simultaneously(with: RotateGesture3D) to combine gestures, use .targetedToEntity() to limit the target, and update the entity’s transform in .onChanged

• What to build: Make a 3D puzzle or assembly tutorial app

  • Why it’s worth doing: SpatialTapGesture’s location3D provides precise 3D click locations. Accurate interaction regions can be configured for complex models via InputTargetComponent and CollisionComponent
  • How to start: Add a collision component to each interactive component, use SpatialTapGesture().targetedToEntity() to capture clicks, and determine which component the user has selected based on the click position

Related Sessions

• Elevate your windowed app for spatial computing — Adapt existing SwiftUI apps to visionOS, learn about windows and ornaments
• Go beyond the window with SwiftUI — Create borderless immersive experiences with ImmersiveSpace
• Build spatial experiences with RealityKit — RealityKit entities - detailed explanation of component system, rendering and animation