Keynote

Highlight

WWDC21 releases iOS 15, iPadOS 15, macOS Monterey, watchOS 8 and tvOS 15; Swift introduces async/await native concurrency syntax, Object Capture API enables minute-level 3D modeling, and Xcode Cloud provides cloud CI/CD services.

Core Content

Online conference continues, developer community expands

Last year’s WWDC attracted nearly 25 million viewers, making it the most-watched event ever.This year it will continue to be free and open to all developers, including 200+ in-depth sessions and one-on-one labs.Apple educates the next generation of developers, especially those underrepresented in the tech world, through Entrepreneurship Bootcamps, Developer Academies (including its latest Detroit Academy), and Learn to Code courses.

（03:11）

iOS 15: Connect, focus, smart, explore

Major update to FaceTime

Spatial audio allows the voices of each person in the call to be heard from the corresponding position on the screen, simulating the sound field of a face-to-face conversation.Voice highlighting uses machine learning to filter out ambient noise and only retains human voices; wide spectrum retains all ambient sounds, making it suitable for music scenes.Portrait mode blurs the background and focuses on the people.

The most crucial one is the FaceTime link - which generates a link that allows Android and Windows users to join the call directly in the browser and still maintain end-to-end encryption.

（05:17）

SharePlay Shared Experience

SharePlay lets users listen to Apple Music, watch videos together, and share screens during FaceTime calls.Everyone’s playback progress is synchronized, and shared playback controls allow anyone to pause or skip songs.SharePlay provides an API that Disney+, Hulu, HBO Max, TikTok, NBA, Twitch, etc. have already connected to.

（09:54）

Notifications and Focus

Notification system redesign: contact photos and larger app icons.Notification summary packages non-urgent notifications, pushes them uniformly at the time you set, and sorts them by priority.Focus mode allows users to create different states for work, personal life and other scenarios. Each state can specify the apps and people that allow notifications, and can even switch to different home screen pages.

（16:52）

Live Text Live Text

Aim the camera at the whiteboard or menu, click the pointer, and text can be selected, copied, and translated.Supports seven languages, including Simplified Chinese and Traditional Chinese.The same set of visual recognition can also identify objects and scenes—types of dogs, varieties of flowers, landmarks, etc.

（20:08）

Wallet and ID

Wallet App adds home keys, company access cards, and hotel keys.The most important thing is proof of identity - some US states support scanning driver’s licenses and state IDs into Wallet, and the TSA will allow digital ID verification at airport security.

（26:25）

iPadOS 15: A productivity leap

Widgets can be freely placed anywhere on the home screen, and extra-large widgets have been added specifically designed for the larger iPad screen.The App Resource Library comes to iPad and automatically categorizes and organizes all apps.

Multitasking operation has been completely revamped.A multitasking menu button is added at the top of each app. Click to select full screen, Split View or Slide Over.The new Shelf area manages all open windows of the App.

Quick Note is the highlight feature of iPadOS 15.Swipe in from the lower right corner of the screen to bring up the floating notes window, which automatically senses the current app and adds corresponding links.

Swift Playgrounds can now develop complete apps.Use SwiftUI to write code, preview the running effect in real time, support code completion and a complete UI component library, and even submit it directly to the App Store.

（37:44）

Privacy: Comprehensive protection from app to network

Mail Privacy Protection prevents email senders from using hidden pixels to track your email opening behavior and IP address.Safari’s Intelligent Tracking Prevention now also hides IP addresses, preventing trackers from correlating your activity across sites.The App Privacy Report shows how often each app accessed sensitive permissions in the past seven days, and which third-party domains they connected to.

Siri now handles voice recognition on the device by default, and does not require a network connection to launch apps, modify settings, control music, etc.

iCloud paid subscriptions are upgraded to iCloud+ at the same price.Private Relay encrypts Safari traffic through a double relay so even Apple can’t know who you are and what websites you visit at the same time.Hide My Email allows you to create a random unique email address and forward it to your personal email.

（49:48）

macOS Monterey: Collaboration across devices

Universal Control is the most amazing demo of this Keynote.Put the iPad next to the MacBook, and the Mac’s mouse cursor will automatically travel to the iPad screen, and one set of keyboard and mouse will control the two devices.Supports up to three devices, and you can directly drag and drop files to transfer across devices.No setup is required.

Shortcuts shortcuts have finally come to Mac, with rich built-in preset shortcuts, and the editor supports drag-and-drop combination operations.Automator continues to be supported and its workflows can be imported into Shortcuts.

Safari redesigns the tab experience.The toolbar is streamlined, the tabs are more compact, and the search bar is merged with the active tabs.Tab groups allow you to save a set of tabs that automatically sync across all your devices.Safari Web Extensions are coming to iPhone and iPad for the first time.

（01:20:10）

Developer Tools: Three major updates

Swift Concurrency — async/await becomes a first-class citizen of the Swift language.Paired with actors technology, concurrent code changes from complex callback hell to a linear sequence of steps.This is the most requested feature from the Swift community.

Object Capture — A revolutionary macOS API that uses photogrammetry to convert a series of 2D photos into photorealistic 3D models in just minutes.Maxon (Cinema 4D), Unity and Wayfair are already plugged in.

Xcode Cloud — Apple’s cloud CI/CD service.After submitting the code, it is automatically built in the cloud, tests run in parallel, and automatically distributed to TestFlight.The build happens in the cloud, and other work can continue locally on your Mac.

（01:37:15）

Detailed Content

SharePlay API Implementation Guide

SharePlay lets your app content be shared with others during FaceTime calls.Implementation based onGroupActivitiesframe.

import GroupActivities

// Define the shareable activity
struct MovieNightActivity: GroupActivity {
    var metadata: GroupActivityMetadata {
        var metadata = GroupActivityMetadata()
        metadata.title = "Movie Night"
        metadata.type = .watchingVideo
        return metadata
    }

    // Add your activity state
    var currentMovie: String?
}

// Start in a FaceTime call
func startActivity() {
    let activity = MovieNightActivity()
    activity.prepareForSharing()

    Task {
        switch await activity.join() {
        case .joined(let session):
            // Joined successfully; start synchronization
            handleJoinedSession(session)
        case .failed:
            // Handle failure
            break
        }
    }
}

Key points:

• GroupActivityProtocols define the types of content that can be shared
• metadataDescribes the activity for display on participant devices
• join()Method attempts to join an existing FaceTime group call
• useGroupSessionMessengerSync data between participants

（09:54）

async/await syntax example

The native concurrency syntax introduced in Swift 5.5 greatly simplifies asynchronous code.

// Old callback approach
func fetchUserProfile(completion: @escaping (Result<User, Error>) -> Void) {
    URLSession.shared.dataTask(with: url) { data, _, error in
        if let error = error {
            completion(.failure(error))
            return
        }
        guard let data = data else { return }
        // Decode data...
        completion(.success(user))
    }.resume()
}

// New async/await approach
func fetchUserProfile() async throws -> User {
    let (data, _) = try await URLSession.shared.data(from: url)
    return try JSONDecoder().decode(User.self, from: data)
}

// Call the async function
Task {
    do {
        let user = try await fetchUserProfile()
        updateUI(with: user)
    } catch {
        handleError(error)
    }
}

Key points:

• asyncKeyword tag asynchronous function
• awaitKeyword wait for asynchronous operation to complete
• throwscan still indicate an error
• TaskCreate an asynchronous execution context
• Code changed from nested callbacks to linear sequence

（01:37:15）

Object Capture API

import RealityKit
import ObjectCapture

func create3DModel(from images: [URL]) async throws -> URL {
    // Create the photogrammetry session
    let session = PhotogrammetrySession()

    // Add images
    for image in images {
        try await session.addImage(at: image)
    }

    // Generate the model
    let outputURL = FileManager.default.temporaryDirectory
        .appendingPathComponent("model.usdz")

    try await session.process(outputURL: outputURL, detail: .full)

    return outputURL
}

Key points:

• PhotogrammetrySessionManage the photogrammetry process
• addImage(at:)Add photos for modeling
• process(outputURL:detail:)Execute modeling
• detailParameters control model accuracy
• It is recommended to provide 30-200 photos from different angles

（01:35:07）

Core Takeaways

1. Connect SharePlay to the video app

• What: Let users watch your video content together during a FaceTime call
• Why it’s worth doing: SharePlay is a natural extension of social consumption and can increase users’ stay time and willingness to share.
• How ​​to start: ImplementationGroupActivityprotocol, useAVPlayerofGroupWatchingActivityto ensure that the playback progress is synchronized

2. Migrate core asynchronous code to async/await

• What: Rewrite callback-based asynchronous operations in the project to async/await syntax
• Why is it worth doing: The code is easier to read and maintain, and the compiler can automatically check for data competition and reduce concurrency bugs
• How ​​to start: Start rewriting from the network request layer and useTaskandTaskGroupManaging structured concurrency

3. Use Object Capture to generate e-commerce 3D display

• What: Provide 3D models of products for users to preview in AR
• Why it’s worth doing: 3D display improves purchasing decision confidence and reduces return rates; Object Capture reduces modeling costs from weeks to minutes
• How ​​to start: Use iPhone to take photos of products from multiple angles, usePhotogrammetrySessionGenerate USDZ and display in AR Quick Look

4. Optimize notification strategy for Focus Mode

• What to do: Review the timing and content of your notifications and distinguish between time-sensitive and non-time-sensitive notifications
• Why it’s worth doing: iOS 15’s Focus Mode may block non-urgent notifications, and reasonable classification can ensure the delivery of important information
• How ​​to get started: Set up the right ones for notificationsinterruptionLevel(time-sensitive can penetrate Focus silence)

5. Set up workflow in Xcode Cloud

• What to do: Migrate CI/CD to Apple’s official cloud service
• Why it’s worth doing: Zero-configuration support for all Apple platforms, deep integration with Xcode, freeing up local machine resources
• How ​​to get started: Enable Xcode Cloud in Xcode’s Product menu, create a workflow and configure trigger conditions

Related Sessions

• Meet async/await in Swift — async/await syntax and structured concurrency model introduced in Swift 5.5
• What’s new in SwiftUI — SwiftUI updates in iOS 15
• What’s new in UIKit — The latest improvements to the UIKit framework and iPad adaptation
• Apple’s privacy pillars in focus — Technical implementation of Apple’s four privacy pillars
• Object Capture in RealityKit — Detailed usage guide of Object Capture API