Highlight
Apple announced Xcode Cloud, Swift Concurrency, new SwiftUI components, RealityKit 2, Focus API, Screen Time API, Widget suggestions and SharePlay at WWDC 2021, allowing developers to complete delivery within Xcode, write concurrent code in Swift, create AR content, and connect applications to system-level collaboration and focused experiences.
Core Content
To make an Apple platform application, writing code only takes up part of the time. The team also needs to run tests, read Pull Requests, sign them, send TestFlights, and receive crash feedback. Previously, these steps were scattered between local scripts, web pages, third-party CI, and App Store Connect, requiring developers to constantly switch environments.
Apple is pulling the delivery link back to Xcode this time. Xcode Cloud is a continuous integration and continuous delivery service built into Xcode 13. It connects to Git source repositories, runs builds and tests, handles signing, hands off builds to TestFlight, and displays reports in Xcode’s Report navigator.
At the language level, Swift 5.5 gets a concurrency model. Developers can useasyncandawaitTo write an asynchronous function, useasync letTo create concurrent subtasks, useactorTo protect mutable state, use@MainActorDeclare APIs that must be executed on the main thread. What used to be accomplished by completion handlers, DispatchQueue, and human conventions is now language rules.
At the UI level, SwiftUI continues to complement common capabilities in application development. List supports sliding operations, pull-down refresh and search, macOS getsTable, accessibility can be improved through rotor and standard control inheritance. Swift Playgrounds 4 allows iPad to create SwiftUI apps and upload them to App Store Connect.
Graphics and AR also lower the barrier to entry. RealityKit 2 introduces Object Capture, which allows developers to take a set of photos with an iPhone or iPad, and then generate a USDZ, USD or OBJ model on the Mac. RealityKit also exposes custom surface shaders, Procedural Geometry API, post processing, compute shaders and geometry modifiers.
The last part is geared toward user relationships. Notifications have interruption level and relevance score, and communication applications can access Focus status. The Screen Time API provides Managed Settings, Family Controls, and Device Activity to parental control applications. Widgets can receive system suggestions through donations of Intents. SharePlay brings FaceTime groups into sharing activities in applications through the GroupActivities framework.
Detailed Content
Xcode Cloud: Putting builds, tests, and TestFlight into Xcode
(02:47) Xcode Cloud is Apple’s new CI/CD (Continuous Integration and Continuous Delivery) service. It’s built into Xcode, supports all Apple platforms, and can put builds, tests, signing, and TestFlight distribution in the cloud.
This talk does not give a command line API. It shows the workflow configuration: select the product, confirm the workflow, authorize the source code warehouse, and connect to App Store Connect. You can then add a Pull Request workflow to allow eachmainThe PR automatically runs the iOS test plan and notifies Slack of the results.
name: Pull Requests
startCondition:
type: pullRequest
targetBranch: main
environment:
xcode: public-beta
macOS: public-beta
actions:
- type: test
testPlan: iOS
simulators: recommended-iPhone-and-iPad
postActions:
- type: notify
destination: team-slack-channelKey points:
nameCorresponds to the Pull Requests workflow created in the lecture. -startConditionIndicates that the workflow is being sent tomainStart on the branch’s Pull Request. -environmentSelect public beta versions of Xcode and macOS for the presentation. -actionsAdd the test action accordingly and select the existing iOS test plan in the project. -simulatorsCorresponds to a set of iPhone and iPad simulators recommended by Xcode Cloud. -postActionsCorresponds to the Notify post action, which notifies the team of success or failure of the build.
(08:21) Xcode Cloud can run multiple test plans in parallel across platforms, device simulators, and OS versions. Xcode 13 also adds Gallery View for viewing UI test screenshots. It also provides Run Test Repeatedly and Expected Fail APIs to help handle occasional failed tests.
import XCTest
final class SmoothieTests: XCTestCase {
func testFavoriteButtonIsUnreliable() throws {
XCTExpectFailure("Favorite button can fail before sign-in is fixed")
let app = XCUIApplication()
app.launch()
app.buttons["Favorite"].tap()
XCTAssertTrue(app.staticTexts["Added to Favorites"].exists)
}
}Key points:
import XCTestIntroducing a testing framework. -SmoothieTestsDefine an XCTest test case. -XCTExpectFailureFlag a known failure and leave the reason for it with the team. -XCUIApplication()Create the application under test. -app.launch()Start the application. -app.buttons["Favorite"].tap()Simulate the user clicking the favorite button. -XCTAssertTrueKeep assertions so that tests can still verify behavior after the problem is fixed.
Swift Concurrency: Write asynchronous processes into straight lines
(21:22) Swift 5.5async/awaitAdd language. For asynchronous functionsasyncMarker, used for call pointsawaitmark. for speechprepareForShowNote: completion handler will turn the process into a nested callback.async/awaitControl flow can be read from top to bottom.
enum StageError: Error {
case missingScenery
}
struct Dancer {}
struct Scenery {}
struct Stage {}
func warmUpCompany() async throws -> [Dancer] {
[Dancer()]
}
func fetchStageScenery() async throws -> Scenery {
Scenery()
}
func setStage(with scenery: Scenery) async throws -> Stage {
Stage()
}
func moveToOpeningPositions(_ dancers: [Dancer], on stage: Stage) async throws {
print("Ready")
}
func prepareForShow() async throws {
let dancers = try await warmUpCompany()
let scenery = try await fetchStageScenery()
let stage = try await setStage(with: scenery)
try await moveToOpeningPositions(dancers, on: stage)
}Key points:
enum StageErrorDefine error types to indicate that asynchronous functions still use Swift’s error mechanism. -warmUpCompany()、fetchStageScenery()、setStage(with:)Use bothasync throwsRepresents asynchronous and may fail. -prepareForShow()It is also an asynchronous function because it calls other asynchronous functions. -try await warmUpCompany()Wait for the dance group to warm up and spread the bug. -try await fetchStageScenery()Wait for the scene to load. -try await setStage(with:)Set the stage with scenery. -try await moveToOpeningPositionsMove the actor position after the first few steps are completed.
(23:05) Structured Concurrency allows concurrent tasks to still belong to the current scope. The speech showedasync let: Warm-up and scene retrieval can be executed in parallel, and can be done when the results are really needed.await。
func prepareForShowInParallel() async throws {
async let dancers = warmUpCompany()
async let scenery = fetchStageScenery()
let stage = try await setStage(with: scenery)
try await moveToOpeningPositions(dancers, on: stage)
}Key points:
async let dancersCreate a subtask and start executionwarmUpCompany()。async let sceneryCreate another subtask and start executionfetchStageScenery().- The two subtasks belong to the same family as the parent task
prepareForShowInParallel()structured scope. -try await setStage(with: scenery)Waiting for the results when the scene needs to be set. -try await moveToOpeningPositions(dancers, on: stage)Waiting for results when dancers are needed.
(24:31) Swift also addsactor. Actors hold state like objects, but external access goes throughasync/awaitCoordinate mutual exclusion. The speech compared it with the previous handwritten serial dispatch queue model and pointed out that actors can reduce data competition caused by forgetting to lock.
actor SmoothieInventory {
private var stock: [String: Int] = [:]
func add(_ name: String, count: Int) {
stock[name, default: 0] += count
}
func take(_ name: String) -> Bool {
guard let count = stock[name], count > 0 else {
return false
}
stock[name] = count - 1
return true
}
}
let inventory = SmoothieInventory()
await inventory.add("Berry Blue", count: 10)
let success = await inventory.take("Berry Blue")
print(success)Key points:
actor SmoothieInventoryDeclare an actor type. -private var stockIt is the mutable state protected by the actor itself. -addModify the state directly inside the actor without the need for handwritten queues. -takeThe inventory is read and updated, and the entire operation is protected by actor isolation. -await inventory.addCalling methods from outside the actor requires asynchronous waiting. -await inventory.takeEnsure that external access will not be destroyed at the same timestock。
(26:30) For UI, Swift provides@MainActor. The speech explained that in the past, every call to an API that must be run on the main thread had to be manually dispatched to the main queue. It is now possible to annotate the main actor on the declaration.
import SwiftUI
@MainActor
final class SmoothieViewModel: ObservableObject {
@Published var title = "Loading"
func updateTitle(_ value: String) {
title = value
}
}Key points:
@MainActorDeclare this view model’s API to run on the main actor. -ObservableObjectLet SwiftUI observe object changes. -@Published var titleNotify the view to refresh when it changes. -updateTitleUI status can be updated directly without handwritingDispatchQueue.main.async。
SwiftUI and Swift Playgrounds: Common interactions become modifiers
(30:14) SwiftUI completes List, Search, Table and Material this year. The Fruta example in the speech puts the collection sliding operation, pull-down refresh, search box and search suggestions into a few lines of code.
import SwiftUI
struct Smoothie: Identifiable {
let id = UUID()
let name: String
let subtitle: String
}
struct SmoothieList: View {
@State private var searchText = ""
@State private var smoothies = [
Smoothie(name: "Berry Blue", subtitle: "Blueberry and banana"),
Smoothie(name: "Green Garden", subtitle: "Kale and apple")
]
var body: some View {
List(filteredSmoothies) { smoothie in
VStack(alignment: .leading) {
Text(smoothie.name)
Text(smoothie.subtitle)
.foregroundStyle(.secondary)
}
.swipeActions {
Button("Favorite") {
print("Favorite \(smoothie.name)")
}
}
}
.refreshable {
await reloadSmoothies()
}
.searchable(text: $searchText) {
Text("Berry").searchCompletion("Berry")
Text("Green").searchCompletion("Green")
}
}
private var filteredSmoothies: [Smoothie] {
guard !searchText.isEmpty else { return smoothies }
return smoothies.filter { $0.name.localizedCaseInsensitiveContains(searchText) }
}
private func reloadSmoothies() async {
smoothies = smoothies
}
}Key points:
@State private var searchTextSave search input. -List(filteredSmoothies)Render filtered data. -.swipeActionsAdd a sliding operation to the list row. -Button("Favorite")Define the favorite button that appears after sliding. -.refreshableAdd pull-down refresh and allow execution of asynchronous functions. -.searchable(text:)Add a system search box, and the platform will determine the location of the search box. -.searchCompletionProvide suggested terms to the search box. -filteredSmoothiesFilter the list based on input.
(31:40) On macOS, SwiftUI increasesTable. The presentation demonstrated using a new file to add a multi-column table and have the search results also be displayed in the table.
import SwiftUI
struct SmoothieTable: View {
let smoothies: [Smoothie]
var body: some View {
Table(smoothies) {
TableColumn("Name", value: \.name)
TableColumn("Description", value: \.subtitle)
}
}
}Key points:
Table(smoothies)Create a macOS multi-column table from a set of data. -TableColumn("Name", value: \.name)BundlenameDisplayed in the first column. -TableColumn("Description", value: \.subtitle)BundlesubtitleDisplayed in the second column.- Tabular data comes from the same set of models and can be reused with search results.
(33:56) Swift Playgrounds 4 supports creating SwiftUI apps on iPad. The speech shows the code on the left, the live preview on the right, code completion below the insertion point, and the console display in real timeprintoutput. Apps can also be uploaded to App Store Connect from App Settings.
import SwiftUI
struct PlaygroundContentView: View {
var body: some View {
Button {
print("Hello from Swift Playgrounds")
} label: {
Label("Say Hello", systemImage: "swift")
}
}
}Key points:
ButtonCreate a clickable control that replaces static text with buttons in the speech. -printThe output will appear in the Swift Playgrounds message bubble and console. -LabelCombine text with SF Symbol. -systemImage: "swift"Use system symbols.
RealityKit 2: Use photos to generate 3D models and then put the models into AR
(40:23) RealityKit 2 adds Object Capture. The process is to first use an iPhone or iPad to take photos from multiple angles, and then use the Object Capture API on Mac to convert the photo folder into an AR-optimized 3D model.
import Foundation
import RealityKit
@main
struct ObjectCaptureTool {
static func main() async throws {
let inputFolder = URL(fileURLWithPath: "/Users/me/CroissantPhotos")
let outputFile = URL(fileURLWithPath: "/Users/me/croissant.usdz")
let session = try PhotogrammetrySession(input: inputFolder)
try session.process(requests: [
.modelFile(url: outputFile, detail: .preview)
])
}
}Key points:
import RealityKitIntroduce the framework where Object Capture is located. -inputFolderPoint to the folder where the photo was taken. -outputFilePoint to the generated USDZ file. -PhotogrammetrySession(input:)Create a photo measurement session that corresponds to “points to the folder of your captured images” in the lecture. -process(requests:)Start model generation, corresponding to “call the process function” in the speech. -.modelFile(url:detail:)Request the generation of a model file. -.previewIt means generating a preview-level detail model. It was mentioned in the speech that you can choose the desired level of detail.
(42:58) After generating the model, you can drag it into the Xcode project, use ARKit to anchor it to the App Clip Code, and then useModelEntityInitialize assets. RealityKit 2 also supports custom surface shaders, Procedural Geometry API, post processing, compute shaders, and geometry modifiers.
import ARKit
import RealityKit
import SwiftUI
struct CroissantARView: UIViewRepresentable {
func makeUIView(context: Context) -> ARView {
let view = ARView(frame: .zero)
let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: [0.2, 0.2]))
if let croissant = try? ModelEntity.loadModel(named: "croissant") {
anchor.addChild(croissant)
}
view.scene.addAnchor(anchor)
return view
}
func updateUIView(_ uiView: ARView, context: Context) {}
}Key points:
ARViewIs RealityKit’s AR rendering view. -AnchorEntity(.plane(...))Create anchor points on a horizontal plane. -ModelEntity.loadModel(named:)Load the 3D model resources in the project. -anchor.addChild(croissant)Hang the model to the anchor point. -view.scene.addAnchor(anchor)Add anchor points to the AR scene. -UIViewRepresentableLet SwiftUI hostARView。
Focus, Screen Time, Widget, and SharePlay: Application access system-level relationships
(52:21) Notification of getting new Interruption Level API. Developers can divide notifications into four categories: passive, active, time sensitive, and critical. The Notification Summary also selects marquee slots at the top of the summary based on the thumbnail and relevance score.
import UserNotifications
func scheduleOrderReadyNotification() async throws {
let content = UNMutableNotificationContent()
content.title = "Smoothie ready"
content.body = "Your Berry Blue smoothie is ready for pickup."
content.interruptionLevel = .timeSensitive
content.relevanceScore = 0.9
let request = UNNotificationRequest(
identifier: "smoothie-ready",
content: content,
trigger: nil
)
try await UNUserNotificationCenter.current().add(request)
}Key points:
UNMutableNotificationContentCreate notification content. -titleandbodyis the text that the user sees. -.timeSensitiveIndicates that notices need to be delivered immediately and speech requests are only used for content that is relevant at the moment and requires immediate attention. -relevanceScoreHelps the system determine presentation priority in snippets. -UNNotificationRequestWrap the content into a notification request. -UNUserNotificationCenter.current().addSubmit notification.
(58:09) Screen Time API is for parental control applications and includes three Swift frameworks: Managed Settings, Family Controls, and Device Activity. Family Controls’ picker returns opaque tokens (opaque tokens), and applications can restrict or observe certain apps and websites, but they cannot get the original bundle ID and URL.
import DeviceActivity
import FamilyControls
import ManagedSettings
struct HomeworkSchedule {
let center = DeviceActivityCenter()
let activity = DeviceActivityName("homework")
func start() throws {
let schedule = DeviceActivitySchedule(
intervalStart: DateComponents(hour: 16),
intervalEnd: DateComponents(hour: 18),
repeats: true
)
try center.startMonitoring(activity, during: schedule)
}
}Key points:
DeviceActivityCenterUsed to register for device activity monitoring. -DeviceActivityName("homework")Name an active window. -DeviceActivityScheduleDefine a time window from 16:00 to 18:00 every day. -repeats: trueIndicates repeated execution. -startMonitoringStart monitoring this active window.- Speech description: After the application receives warning and completion events, it can adjust restrictions or encourage children to complete homework.
(01:04:48) Widget gains suggestion capabilities. The system can recommend a widget into the Smart Stack based on the user’s interaction with the application and new relevant information donated by the application through Intents. The example in the presentation was the Fruta app recommending widgets to users who often order green juice in the morning.
import Intents
func donateMorningSmoothieInteraction() {
let intent = INIntent()
let interaction = INInteraction(intent: intent, response: nil)
interaction.identifier = "morning-green-juice"
interaction.donate { error in
if let error {
print("Donation failed: \(error)")
}
}
}Key points:
INIntentRepresents an intent that can be donated to the system. -INInteractionWrap intent into a user-related interaction. -identifierGive this donation a stable flag. -donateLeave the interaction to the system for scenarios such as widget suggestions.- The speech pointed out that both past usage behavior and new related intent donation can help the system suggest widgets.
(01:06:38) SharePlay is driven by the GroupActivities framework. The developer defines aGroupActivity, the system brings the same group of people into the app after a user starts an activity during a FaceTime call. Media applications can use AVPlayer’s playback coordinator to achieve synchronous playback, and custom applications can useGroupSessionMessengerSend real-time data.
import AVFoundation
import GroupActivities
struct WatchSmoothieShow: GroupActivity {
let url: URL
var metadata: GroupActivityMetadata {
var metadata = GroupActivityMetadata()
metadata.title = "Watch Smoothie Show"
metadata.type = .watchTogether
return metadata
}
}
func startSharedPlayback(url: URL, player: AVPlayer) async throws {
let activity = WatchSmoothieShow(url: url)
let activation = try await activity.prepareForActivation()
guard activation == .activationPreferred else {
player.play()
return
}
_ = try await activity.activate()
}Key points:
WatchSmoothieShowobeyGroupActivity, defines an activity that can be shared. -urlIt is the content address to be loaded by each participant. Applications that already have deep links in the speech description can reuse it. -metadataProvide the title and activity type required by the system UI. -prepareForActivation()Let the system display a confirmation dialog; if the user is not in a FaceTime call, it will return immediately. -.activationPreferredIndicates that the system recommends initiating sharing activities. -activate()Actually start the SharePlay activity. -player.play()Is the local playback path when the sharing activity is not started.
import AVFoundation
import GroupActivities
func coordinatePlayback(player: AVPlayer, session: GroupSession<WatchSmoothieShow>) {
player.playbackCoordinator.coordinateWithSession(session)
session.join()
}Key points:
GroupSession<WatchSmoothieShow>Represents the current SharePlay session. -playbackCoordinatorIt is the playback coordination entrance of AVPlayer. -coordinateWithSession(session)Connect the player to a Group Activities session to achieve frame-level synchronization. -session.join()Let this device join the sharing activity.
Core Takeaways
-
What to do: Connect the team application to the PR testing process of Xcode Cloud. Why it’s worth doing: The talk showcases the Pull Request workflow, parallel testing, build reports, and Slack notifications to get team feedback back into Xcode. How to start: Create a workflow in the Xcode Cloud menu of Xcode 13 and set the Pull Request to
mainWhen starting, add test action, select test plan, and then add Notify post action. -
What to do: Migrate a network request-intensive page to Swift Concurrency. Why it’s worth doing:
async/await、async letand actors can reduce callback nesting and leave shared state protection to the language model. How to start: First encapsulate the completion handler intoasync throwsfunction, reuseasync letIndependent data is pulled in parallel; shared cache is put into actors. -
What to do: Add pull-down refresh, search suggestions and sliding collection to the list page. Why it’s worth doing: SwiftUI turns these common interactions into
.refreshable、.searchable、.swipeActions, can reduce custom UIKit wrapping. How to start: Make the data modelIdentifiable,existListAdd on line.swipeActions, add to the outside of the list.refreshableand.searchable(text:)。 -
What to do: Generate AR product previews for e-commerce or menu applications. Why it’s worth doing: Object Capture can convert photos of real objects into USDZ, and RealityKit 2 can display models in App Clips or full applications. How to get started: Take photos of objects from all angles with iPhone and use them on Mac
PhotogrammetrySessionGenerate USDZ and then useModelEntity.loadModel(named:)Put into AR scene. -
What to do: Add SharePlay to a video, whiteboard, or reading app. Why it’s worth doing: GroupActivities provides FaceTime groups, synchronous playback, and end-to-end encrypted data channels, so applications don’t need to build their own real-time collaboration infrastructure. How to get started: Define a compliance
GroupActivityactivity type; for media playbackAVPlayer.playbackCoordinator.coordinateWithSession, for custom collaborationGroupSessionMessengerSend operation data.
Related Sessions
- Meet Xcode Cloud — Introduce the build, test, signing and distribution process of Xcode Cloud from a product perspective.
- Meet async/await in Swift — In-depth explanation
async、await, async properties and how to migrate from completion handlers. - What’s new in SwiftUI — The system expands SwiftUI’s new capabilities such as List, Search, Table, Material, and Canvas.
- Create 3D models with Object Capture — Detailed instructions on how to take photos and generate 3D models with RealityKit.
- Meet Group Activities — Explains SharePlay’s GroupActivity, session management, and shared experience architecture.
Comments
GitHub Issues · utterances