Keynote

Highlight

WWDC 2023 introduced Apple Vision Pro and visionOS, completing the Apple Silicon transition with M2 Ultra Macs, while delivering major updates to iOS 17, iPadOS 17, macOS Sonoma, and watchOS 10.

Core Content

WWDC 2023 was Apple’s most important developer conference in nearly a decade. Three major things happened at once: the Mac lineup completed its Apple Silicon transition, six software platforms got annual major updates, and the entirely new Apple Vision Pro hardware platform was born. Developers need to focus not just on new APIs, but on a shift in computing paradigm — from desktop to mobile to spatial computing.

Mac: Full Apple Silicon Lineup Complete

The Mac’s Apple Silicon transition began with M1 in 2020. Three years later, the final piece — Mac Pro — finally got Apple’s own chip.

(04:55) 15-inch MacBook Air. The first 15-inch MacBook Air. 11.5mm thick, about 3 pounds, M2 chip, 18-hour battery life. 15.3-inch Liquid Retina display, 500 nits brightness, 1 billion colors. Starting at $1299; 13-inch model reduced to $1099.

(09:49) Mac Studio update. New M2 Max and M2 Ultra options. M2 Max version is 25% faster than M1 Max; After Effects rendering 50% faster.

(10:42) M2 Ultra chip. Two M2 Max chips connected via UltraFusion architecture: 24-core CPU (20% faster than M1 Ultra), 76-core GPU (30% faster), 32-core Neural Engine (40% faster), up to 192GB unified memory. Can train large transformer models on a single machine.

(13:35) Mac Pro with Apple Silicon. M2 Ultra, 6 PCIe Gen 4 expansion slots, 8 Thunderbolt ports. Built-in Media Engine delivers video processing equivalent to 7 Afterburner cards. Starting at $6999.

(18:24) Contact Posters. Users customize the avatar, font, and colors shown on incoming calls. Third-party VoIP apps can use them via CallKit.

(19:39) Live Voicemail. Real-time transcription of voicemail messages displayed on screen while the caller leaves a message; users can answer anytime. Transcription is fully on-device.

(21:01) Messages updates. Search supports multi-condition filtering; audio messages auto-transcribe; Check In automatically notifies family you’ve arrived safely; all Emoji become stickers; create Live Stickers from photo subjects, usable system-wide.

(25:43) NameDrop. Two iPhones brought together exchange contact info. Also works between iPhone and Apple Watch.

(26:08) AirDrop upgrade. Bringing two devices together starts transfer or SharePlay. After exceeding AirDrop range, large files can continue transferring over the internet.

(27:25) Keyboard and dictation. Auto-correction upgraded to a Transformer language model with sentence-level grammar correction. Predictive text shown inline; press space to complete a word or sentence. Dictation also upgraded to a Transformer-based speech recognition model.

(29:45) Journal app. New app using on-device machine learning to intelligently generate writing suggestions from photos, location, music, and more. Provides a Suggestions API for third-party apps.

(32:05) StandBy. Full-screen display mode when iPhone is placed horizontally while charging — clocks, photo slideshows, and widgets.

iPadOS 17: Personalization and Productivity

(35:49) Interactive widgets. Widgets can now be directly interactive — mark reminders complete, control HomeKit devices, play music without opening apps. Third-party apps can add interactive features too.

(36:50) Lock Screen personalization. iPad gets the same lock screen customization as iPhone: wallpaper, fonts, colors, widgets. Live Activities come to iPad lock screen.

(39:11) Health app on iPad. HealthKit sync extends to iPad; developers can build health and fitness apps for the large screen.

(40:27) PDF enhancements. System automatically recognizes PDF form fields and supports AutoFill. Notes app can display, annotate, and collaborate on PDFs in-app.

macOS Sonoma: Widgets, Gaming, and Productivity

(45:11) Desktop widgets. Widgets are no longer limited to Notification Center — place them directly on the desktop. Via Continuity, Mac can access iPhone widgets without installing the corresponding app on Mac.

(48:15) Game Mode. Automatically assigns highest CPU/GPU priority to games. AirPods audio latency significantly reduced; Xbox/PlayStation controller Bluetooth sample rate doubled.

(49:03) Game Porting Toolkit. New Metal tool to quickly evaluate and port games from other platforms to Mac. Assessment that used to take months now takes days.

(51:54) Video conferencing enhancements. Presenter Overlay places the presenter on top of shared content. Reactions video effects triggered by gestures.

(53:15) Safari updates. Private browsing windows auto-lock; block known trackers; strip URL tracking parameters. Support secure group sharing of passwords and passkeys. Profiles separate work and personal browsing data. Web Apps can add any website to the Dock.

Audio and Home

(57:50) AirPods Adaptive Audio. Dynamically blends noise cancellation and transparency. Conversation Awareness lowers music volume when you start speaking.

(01:03:05) FaceTime on Apple TV. Continuity Camera wirelessly connects iPhone/iPad cameras to Apple TV. Continuity Camera API open to developers; Zoom and Webex coming to tvOS by year end.

watchOS 10: Redefining Interaction

(01:07:00) Smart Stack. Rotate the Digital Crown from any watch face to reveal a widget stack; machine learning shows relevant info at the right time.

(01:10:27) Cycling features. Apple Watch connects to Bluetooth bike sensors. Auto-estimates FTP and provides Power Zones training data. Cycling workouts automatically display as Live Activity on iPhone.

(01:13:37) Golf/tennis workout APIs. Apple Watch Series 8 and Ultra high-frequency motion sensors detect rapid direction and acceleration changes; developers access data via API.

(01:15:02) Mental health. Mindfulness app can log emotions and moods. Health app provides standardized depression and anxiety assessment scales.

(01:17:04) Vision health. Apple Watch measures outdoor daylight time. Screen Distance uses TrueDepth camera to detect if device is too close to eyes.

Apple Vision Pro: Spatial Computing Platform

(01:21:05) The centerpiece of the entire Keynote. Apple Vision Pro is a standalone spatial computer starting at $3499, shipping early 2024 in the US.

(01:24:22) Interaction. Three inputs: eyes (look to select), finger pinch to choose, voice dictation and Siri. No controllers required.

(01:48:16) Display system. Two micro-OLED panels, each stamp-sized, with more pixels per eye than a 4K TV (23 million total). Each pixel is only 7.5 microns wide.

(01:51:04) Dual-chip architecture. M2 handles computation; new R1 chip processes sensor data. R1 delivers new images to displays in 12 milliseconds — 8x faster than a blink.

(01:28:25) EyeSight. External curved OLED shows the wearer’s eyes to people nearby. Light effects indicate full immersion.

(01:52:50) Digital Persona. Encoder-decoder neural network generates the user’s digital likeness, dynamically matching face and hand movements. Communicates with over a billion FaceTime devices.

(01:53:28) visionOS. Built on macOS/iOS/iPadOS with a real-time execution engine, dynamic foveated rendering pipeline, and first multi-app 3D engine.

(01:54:51) Developer ecosystem. Build with familiar tools: Xcode, SwiftUI, RealityKit, ARKit. Reality Composer Pro assembles 3D scenes. Hundreds of thousands of iPad/iPhone apps run at launch. Deep Unity integration.

(01:58:46) Privacy. Optic ID uses iris recognition. Gaze tracking data is isolated from apps — apps only know click results, not gaze trajectory. Camera data processed at system level.

Detailed Content

visionOS Development Framework Stack

Apple Vision Pro’s developer toolchain builds on existing technology:

Layer	Technology	Purpose
UI framework	SwiftUI	Windows, volumes, spatial interfaces
3D framework	RealityKit	3D rendering, animation, physics, audio
Spatial awareness	ARKit	Hand tracking, scene understanding, plane detection
Low-level rendering	Metal / CompositorServices	Custom rendering pipelines, fully immersive experiences
Content tools	Reality Composer Pro	Assemble, edit, preview 3D scenes
Game engine	Unity	Port existing Unity VR/3D games

Key points:

SwiftUI extends WindowGroup, Volume, and ImmersionSpace scene types
RealityKit’s RealityView embeds 3D content in the SwiftUI view hierarchy
ARKit hand tracking and scene understanding require Full Space permission
Existing iPad/iPhone apps run on visionOS via a compatibility layer

visionOS Privacy Architecture

┌─────────────────────────────────────────┐
│  App Process                            │
│  - Can only receive tap results         │
│  - Cannot access gaze trajectory        │
│  - Cannot access raw camera data        │
└─────────────────────────────────────────┘
                    ↑
┌─────────────────────────────────────────┐
│  System Process (isolated)              │
│  - Gaze tracking data processing        │
│  - Camera data processing               │
│  - Optic ID iris recognition            │
└─────────────────────────────────────────┘

Key points:

Gaze tracking data is processed in an isolated system process
Apps only receive “user tapped this element” results
Camera data is processed at system level; apps cannot see surroundings
Optic ID data is encrypted and stored in the Secure Enclave

Core Takeaways

1. Three strategies to prepare your app for visionOS

Evaluate which visionOS form fits your app — windowed (Windows), volumetric (Volumes), or full spatial (Spaces). iPad/iPhone apps run on Vision Pro launch day, but only visionOS-adapted apps unlock spatial computing advantages. Entry point: SwiftUI WindowGroup and Volume scene types; learn RealityView and Model3D.

2. Boost engagement with interactive widgets

Add interactivity to existing widgets — not just display info, but let users act directly. iPad and Mac widgets both support interaction; users complete tasks without opening apps. Entry point: Button and Toggle in widgets via SwiftUI; background operations via App Intents.

3. Prepare for Mac game porting

Evaluate porting games to macOS; use Game Porting Toolkit for quick validation. Game Mode provides out-of-the-box performance optimization for all Mac games. Entry point: download Game Porting Toolkit and run the evaluation tool; learn Metal Shader Converter.

4. Build a spatial computing prototype

Use Xcode’s visionOS templates and simulator to quickly build a spatial prototype. Even if your main app is 2D, try adding a simple 3D element like a rotatable product model. Entry point: Xcode 15 visionOS template + Reality Composer Pro + simulator preview.

Build great games for spatial computing — Framework choices and immersion level design for visionOS game development
Meet SwiftUI for spatial computing — Build spatial user interfaces with SwiftUI on visionOS
What’s new in SwiftUI — New SwiftUI features and APIs from WWDC 2023
What’s new in Swift — Latest advancements in the Swift programming language