WWDC Quick Look đź’“ By SwiftGGTeam
中文 EN 日本語
Build immersive web experiences with WebXR

Build immersive web experiences with WebXR

WWDC 2024
Watch original video

Highlight

Safari on visionOS 2.0 officially supports WebXR immersive VR sessions—web pages can deliver fully immersive 3D experiences on Vision Pro without App Store review or requiring users to download an app.

Core Content

Building a 3D experience on Vision Pro traditionally meant writing a native app—registering a developer account, passing review, and waiting for users to download. The barrier is high and reach is slow. Many teams already have 3D content on their websites (product showcases, virtual galleries, data visualizations), but it is constrained by 2D browsing and cannot let users “step inside” to feel space and scale.

Safari on visionOS 2.0 introduces WebXR support and changes that. WebXR is a W3C web standard for launching immersive XR sessions in the browser. The user taps a button, Safari shows a permission prompt, and then WebGL content in the page fills the entire field of view—fully immersive, with the same effect as a native VR app. Exit by pressing the Digital Crown or returning to the Home Screen.

The key advantage: no App Store, no installation—just open a web page. For sites that already have WebGL content, the incremental work is small—check whether navigator.xr is available, add an “Enter VR” button, and call requestSession to switch to immersive mode. visionOS also adds a transient pointer input type specifically for WebXR, mapping the natural “look and pinch” interaction while strictly protecting gaze privacy—websites only receive gaze direction and hand position at the moment of a pinch.

Detailed Content

Security and Privacy

WebXR’s security design runs through the entire stack. (03:27)

  • HTTPS is mandatory: All WebXR content must be served over HTTPS to prevent man-in-the-middle injection.
  • iFrames need allow="xr-spatial-tracking": Prevents third-party code (such as ads) from starting a VR session without the developer’s permission.
  • User-initiated entry: There must be a button or other interactive element for the user to start an XR session—automatic entry on page load is not allowed.
  • System exit gestures are reserved: The Digital Crown and Home Screen gestures are always used to exit; developers cannot intercept them.
  • Gaze privacy: On visionOS, spatial input exposes gaze direction only at the moment of a pinch; hand position is also visible only during the pinch.

Building Immersive Scenes Quickly with A-Frame

A-Frame is a declarative HTML-based WebGL framework well suited for WebXR beginners. The following code renders a 3D scene with sky and ground on Vision Pro and supports look-and-pinch interaction. (09:02)

<html>
  <head>
    <script src="https://aframe.io/releases/1.5.0/aframe.min.js"></script>
  </head>
  <body>
    <a-scene cursor="rayOrigin: mouse; fuse: false">
      <a-box position="0 1.6 -3" color="red"
             animation="property: rotation; to: 0 360 0; loop: true; dur: 5000">
      </a-box>
      <a-sphere position="1 1.6 -3.5" color="blue"
                animation="property: position; to: 1 2.0 -3.5; dir: alternate; loop: true; dur: 1000">
      </a-sphere>
      <a-cylinder position="-1 1.6 -3" radius="0.3" height="1" color="green"></a-cylinder>
      <a-plane position="0 0 -3" rotation="-90 0 0" width="20" height="20" color="#7BC8A4"></a-plane>
      <a-sky color="#ECECEC"></a-sky>
    </a-scene>
  </body>
</html>

Key points:

  • <a-scene> is A-Frame’s root element and automatically handles WebGL rendering and WebXR session management.
  • The cursor component uses a raycaster to detect gaze targets and fires click, mouseenter, and mouseleave events on pinch.
  • 3D coordinates use meters—WebXR uses real-world scale, so position="0 1.6 -3" places an object at eye height, 3 meters in front.
  • The animation component adds keyframe animation to objects without writing JavaScript.
  • A-Frame includes a built-in “Enter VR” button that automatically calls requestSession when tapped.

WebXR Lifecycle

The full WebXR session lifecycle has four steps. (10:48)

// 1. Check support
if (navigator.xr) {
  const supported = await navigator.xr.isSessionSupported("immersive-vr");
  if (supported) {
    // Show the "Enter VR" button
  }
}

// 2. Request a session
const session = await navigator.xr.requestSession("immersive-vr", {
  requiredFeatures: ["local-floor"],
  optionalFeatures: ["hand-tracking"]
});

// 3. Set up the render loop
const gl = canvas.getContext("webgl2");
const glLayer = new XRWebGLLayer(session, gl);
session.updateRenderState({ baseLayer: glLayer });
const refSpace = await session.requestReferenceSpace("local-floor");

session.requestAnimationFrame(function onFrame(time, frame) {
  const pose = frame.getViewerPose(refSpace);
  if (pose) {
    // Render the left and right eyes with pose.views
  }
  session.requestAnimationFrame(onFrame);
});

// 4. End the session
session.addEventListener("end", () => {
  // Show the "Enter VR" button again
});

Key points:

  • isSessionSupported("immersive-vr") checks whether the current device supports immersive VR and returns a Promise with a boolean result.
  • The second argument to requestSession is a feature list: if features in requiredFeatures are unavailable or denied by the user, the entire request fails; features in optionalFeatures are simply omitted when unavailable and do not cause failure. Prefer optional when possible.
  • The local-floor reference space places the coordinate origin near the user’s feet, suitable for standing experiences.
  • WebXR’s requestAnimationFrame is attached to the session rather than window, because XR device refresh rates may differ from the display.
  • getViewerPose(refSpace) returns the viewer pose for the current frame, including left and right eye view matrices for WebGL rendering.
  • When the session ends (user presses the Digital Crown or the developer calls session.end()), the end event fires and you can show the entry button again.

Transient Pointer: Natural Input on visionOS

visionOS’s “look and pinch” interaction is mapped to transient pointer in WebXR. (15:19)

Traditional XR headsets use tracked pointers (continuously tracked controllers or hands), while transient pointer exists only when the user performs a pinch gesture. The full flow:

  1. Initial state: The website does not know where the user is looking or where their hands are.
  2. User pinches: A new input source is added to session.inputSources with targetRayMode set to "transient-pointer". This triggers inputsourceschange and selectstart events.
  3. User moves hand: Grip space follows the contact point between thumb and index finger; target ray space follows hand movement (rather than continuously tracking gaze direction).
  4. User releases: The select event fires (indicating a successful selection), then selectend. The input source is removed and the website can no longer perceive gaze or hands.

This design trades privacy for experience: websites only receive spatial information when the user explicitly interacts.

Hand Tracking

If you need continuous hand tracking, add "hand-tracking" to the feature list when requesting a session. (20:06)

With hand tracking enabled, detected hands appear in inputSources as tracked pointers. Each hand includes 25 joint spaces that you can render with WebGL. Note: with hand tracking enabled, you must render the user’s hands yourself—the system no longer shows the real hands. Transient pointer remains available when hand tracking is on—the same experience can support both close-range precise manipulation (hand tracking) and long-range selection (look and pinch). When both hands pinch simultaneously, up to 4 input sources may appear.

Testing and Debugging

During development, you can test WebXR in the visionOS Simulator. (23:21)

  • Start an HTTP server on localhost (localhost is a secure context and does not require HTTPS).
  • Use WASD keys to move, right-click drag to rotate the view, and mouse click to simulate transient pointer.
  • Use Web Inspector in macOS Safari to remotely debug pages in the simulator: view the console, set breakpoints, and step through JavaScript.
  • Hand tracking cannot be tested in the simulator; you need a physical device.

Core Takeaways

  • What to build: Add a WebXR 3D preview mode to e-commerce product pages. After the user taps a “View in Space” button, the product model appears in Vision Pro at real-world scale, and the user can walk around and inspect details up close. Why it’s worth doing: Vision Pro users can perceive real product size and presence, which is far more intuitive than 2D images, without building a native app. How to start: Wrap an existing 3D model with A-Frame’s declarative markup, add an <a-scene> and model-loading component, and show the entry button after checking navigator.xr.isSessionSupported("immersive-vr").

  • What to build: Add an immersive viewing mode to data visualization dashboards. Users can browse charts in 3D space and observe data relationships from different angles. Why it’s worth doing: Spatial computing devices are naturally suited for multi-dimensional data—each dimension can map to a spatial axis, and users “see” data by moving their head. How to start: Use Three.js to map existing Chart.js/D3 chart data to 3D geometry, use WebXR’s local-floor reference space to place charts on the ground, and use transient pointer interaction to select data points.

  • What to build: Build interactive 3D teaching scenes for educational websites—chemical molecules, human anatomy, architectural models—where students can rotate, disassemble, and assemble models in space. Why it’s worth doing: 3D educational content on flat screens is always limited by viewpoint and occlusion; in space, users can observe freely and manipulate with their hands, significantly improving learning efficiency. How to start: Load existing glTF/GLB models with Babylon.js, enable hand-tracking so users can directly manipulate model parts, and use transient pointer for long-range selection and menu operations.

Comments

GitHub Issues · utterances