WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Design no-code games with Reality Composer Pro 3

Design no-code games with Reality Composer Pro 3

Reality Composer ProScript GraphvisionOSSwiftUIRealityKit
Watch original video

Highlight

Reality Composer Pro 3’s Script Graph, a visual node scripting system, lets designers build spatial interaction prototypes for visionOS without writing Swift code, and supports live parameter tuning on a real Vision Pro device.

Core Content

From “It Moves Only After Code” to “Connect Nodes and Play”

In the past, when creating a visionOS prototype, designers would finish sketches and then programmers had to write a pile of RealityKit code to implement gesture responses, physics collisions, and animation triggers. After testing on Vision Pro, they might discover that drag sensitivity felt wrong, return to Xcode, change a number, rebuild, put the headset back on, and repeat. The loop was extremely inefficient.

Reality Composer Pro 3 changes that workflow with Script Graph. Script Graph is a node-based visual scripting system that builds interactions through event-driven logic. You do not need to write a single line of code. By dragging nodes and connecting wires in the editor, you can listen for gesture events, change entity properties, and drive the physics engine.

(01:02)

Live Preview Makes “Feel” Tunable

The hardest part of spatial interaction is that “feel cannot be predicted on a 2D screen.” A parameter that feels right when dragging with a mouse on a Mac can feel completely different when you pinch and throw with your hand in Vision Pro.

Variables in Script Graph can be made public and exposed on the entity’s Scripting Component. Combined with Live Preview, coming later this year, designers can wear Vision Pro and directly change parameter values on a Mac virtual display. In the demo, the presenter adjusts the drag speed multiplier from 1.3 to 1.5, then to 1.1, and finally settles on 1.15, all without recompiling.

The more important piece is the Override mechanism: the same Script Graph can be reused across multiple entities, and each entity can have its own parameter override values. That means a scene can contain multiple “nuts” that share the same drag logic while each has a different response speed.

(05:37)

The Physics Engine Can Also Be Node-Based

If dragging an object only sets its position directly, the interaction can feel floaty. Script Graph can connect to RealityKit’s physics engine: add a Physics Body Component to the entity, then use an Add Force node to convert drag displacement (dragDelta) into force applied to the object.

In the demo, the presenter uses a Set Variable node to store the previous frame’s drag position, subtracts it from the current position to get displacement, and multiplies that by a coefficient before feeding it into Add Force. The object gains inertia and a projectile-like motion, so it can be “thrown.”

At the same time, a Set PhysicsBodyComponent node can disable gravity and increase linear damping when dragging starts, making the object easier to control, then restore gravity when it is released. All physics interaction tuning happens inside the node graph.

(08:34)

Custom Events Connect 3D and SwiftUI

When the player steals the squirrel’s nut, the squirrel needs to complain. The trigger for “speaking” is handled in Script Graph, but the speech bubble UI is built with SwiftUI.

Script Graph provides a Send Scene Event node that can send custom events with parameters at the scene level. In Swift code, you listen with scene.subscribe(forEventName:), update state variables, and then mount SwiftUI views onto 3D entities through RealityView attachments.

This design lets designers arrange “when to trigger” in the node graph, while programmers implement “what the UI looks like” with SwiftUI. The two sides are decoupled through events.

(16:04)

Details

Make an Entity Draggable: Basic Node Combination

Drag support requires three components:

  • Input Target Component: makes the entity a gesture target
  • Collision Component: defines the collision volume for gesture detection
  • Hover Effect Component: shows highlighted feedback when gazed at

Then, in Script Graph, use the On Drag event node to listen for drag gestures and connect the event’s Scene Location output to the translation input of the Set Transform node. Every time a drag event fires, the entity’s position updates to the spatial coordinates of the gesture.

(02:55)

Tune Parameters with Variables and Math Nodes

The Scene Translation output from On Drag can drive position directly, but the feel may be poor. In the demo, a Multiply by Number node scales the displacement, controlled by a public Input variable called dragSpeed.

When creating an Input variable in the Inspector, you can set its type (Number), default value (1.3), and access level (public). Once set to public, the variable appears in the entity’s Scripting Component panel, making it easy to tune during real-device testing.

(06:28)

Prototyped Subgraph: Reuse Logic

When the node graph grows complex, you can package a group of nodes into a single node with Subgraph. Select the related nodes, right-click, choose Compose Subgraph, and name the subgraph.

If the logic needs to be reused across multiple Script Graphs, convert it further into a Prototyped Subgraph. Once converted, the subgraph appears in the project asset browser and is available as a standard node in the Add Node menu. The demo packages the logic for “detect whether a Boolean value has changed” into a reusable node.

(11:32)

Custom Event Library: Cross-Entity Communication

When two entities need to communicate, create a Custom Event. First, create a Custom Node Library in the Project Browser, add a Custom Event inside it and name it, such as nutIsDragged, then add properties, such as nutPosition with type Vector. After clicking Sync Nodes, the custom event appears in the node menu.

The sender uses the Send “nutIsDragged” node to send the Nut’s world coordinates as a parameter. The receiver uses the On “nutIsDragged” event node to listen and uses the received nutPosition to drive its own rotation or material changes.

(13:27)

Trigger SwiftUI from Script Graph

Send a scene event from Script Graph:

// Listen for scene events in SwiftUI and update state
if let scene = entity.scene {
    scene.subscribe(forEventName: "squirrelTalk", on: { event in
        if let sayThis: String = try? event.value("sayThis") {
            self.sayThis = sayThis
        }
    }).store(in: &cancellables)
}

Key points:

  • subscribe(forEventName:on:) registers a listener for the scene event with the specified name
  • event.value("sayThis") extracts the event parameter by name; the type must match the type defined in Script Graph
  • .store(in: &cancellables) stores the subscription in a set of Combine cancellation tokens to avoid memory leaks

Mount a SwiftUI view onto a 3D entity:

RealityView { content, attachments in
    // Load 3D scene content
} attachments: {
    Attachment(id: "squirrelTalk") {
        SquirrelTalkAttachmentView(text: sayThis)
    }
}

Key points:

  • The attachments closure defines SwiftUI views that can be attached to 3D entities
  • The Attachment(id:) ID must match the ID bound in code through entity.components.set(AttachmentComponent)
  • SquirrelTalkAttachmentView(text:) receives the text parsed from the scene event and renders the speech bubble

(17:15)

Key Takeaways

1. Quickly Validate Spatial Interaction Prototypes

  • What to do: Use Script Graph to build a draggable 3D interaction prototype with physics feedback in 30 minutes, then test its feel directly on Vision Pro.
  • Why it is worth doing: You can validate whether the core interaction works without writing Swift code, making failure cheap.
  • How to start: Import a 3D model into Reality Composer Pro 3, add Input Target, Collision, and Hover Effect components, and create a Script Graph that connects On Drag to Set Transform.

2. Add Physics Feedback to an Existing App

  • What to do: Add inertia and projectile motion to interactive objects in a visionOS app.
  • Why it is worth doing: Directly setting position feels fake. Physics motion driven by Add Force gives interactions a sense of weight.
  • How to start: Add a Physics Body Component to the entity, use Set Variable in Script Graph to store the previous frame position, calculate dragDelta, scale the force with Multiply by Number, and feed it into Add Force.

3. Use SwiftUI for HUDs and Dialogue Systems in 3D Scenes

  • What to do: Use SwiftUI inside a RealityKit scene to implement health bars, speech bubbles, mission hints, and other 2D UI.
  • Why it is worth doing: SwiftUI’s declarative syntax is much faster than building low-level UI directly in RealityKit, and existing components can be reused.
  • How to start: Trigger events in Script Graph with Send Scene Event, listen in Xcode with scene.subscribe, and bind SwiftUI views to the corresponding entities through RealityView attachments.

4. Build a Reusable Interaction Component Library

  • What to do: Package common interaction logic, such as “double-tap to zoom,” “gaze highlight,” or “drag and snap,” into Prototyped Subgraphs and reuse them across projects.
  • Why it is worth doing: It avoids wiring nodes from scratch every time and keeps interactions consistent across projects.
  • How to start: Create a Custom Node Library in the project, convert validated Subgraphs into Prototyped Subgraphs, and let team members call them directly from the Add Node menu.

5. A Hybrid Workflow for Designers and Programmers

  • What to do: Let designers own interaction logic and event timing in RCP3, while programmers own UI implementation and the data layer in Xcode.
  • Why it is worth doing: Designers can iterate on interactions without waiting for engineering scheduling, and programmers do not need to recompile for every small adjustment.
  • How to start: Agree on Custom Event naming conventions and parameter types. Designers arrange event flows in Script Graph, and programmers write event listeners and UI rendering code.

Comments

GitHub Issues · utterances