Platforms State of the Union

Highlight

Platforms State of the Union at WWDC 2022 strung together Xcode Cloud online, Swift 5.7 syntax and concurrency extensions, SwiftUI new navigation and layout, WidgetKit lock screen capabilities, App Intents, Passkeys, Metal 3, WeatherKit and VisionKit into a platform roadmap.

---

Core Content

If a team is upgrading an Apple platform app in 2022, the question isn’t just “what APIs are coming this year?” The real trouble comes from simultaneous changes on multiple levels: the code needs to be easier to maintain, the interface needs to cover more devices, the system entrance has expanded from the home screen to the lock screen, Siri, Spotlight, and Messages, and underlying capabilities such as graphics and maps are also advancing.

The purpose of this State of the Union is to put these changes back into the same picture. Susan Prescott first announced that Xcode Cloud was available that day and explained that it integrates TestFlight, App Store Connect, mainstream Git source code hosting, and REST APIs. The speech then started from Swift, SwiftUI, system integration, security, graphics, maps, weather, and visual recognition.

What developers get is a set of paths that can be strung together. Swift and SwiftUI reduce daily development costs; WidgetKit, Live Activities, App Intents, and Messages Collaboration API bring functions to the system surface; Passkeys, Metal 3, WeatherKit, and VisionKit handle more specific issues such as login, graphics performance, weather data, and visual recognition respectively.

Reduce friction starting with the tool chain

The positioning of Xcode Cloud is straightforward: put continuous integration and delivery into Xcode and host it in the cloud. It supports all Apple platforms, connects to TestFlight and App Store Connect, and can also be connected to the team’s existing processes through REST API. For small teams without full CI/CD infrastructure, this means a shorter path from code submission to test distribution.

Swift Playgrounds 4 is also placed in the same toolchain story. It can already build apps, submit them directly to the App Store, and supports SwiftUI. Prototyping, learning, and submitting form a lightweight closed loop.

Swift and SwiftUI continue to be the main line

Josh Shaffer identified the next generation combination of Apple platforms as Swift, SwiftUI, and Xcode Previews. Swift 5.7 brings Async Algorithms, distributed actors, Regex,someandanyGeneric writing, and Swift Package plugins. SwiftUI adds navigation, layout, Share Sheet, UIKit collection view cell hosting and other capabilities.

The Food Truck example is used throughout several paragraphs. It first uses Swift Charts to display donut sales data, and then usesViewThatFitsHandle horizontal and vertical screen layouts, use new navigation split view andNavigationStackMake iPad navigation, and finally use the same set of SwiftUI code to bring the app to Mac and add menu bar extra.

There are more system entrances, and application functions should be more discoverable.

The iOS 16 lock screen expands the value of WidgetKit. The three lock screen widget families, Circular, Rectangular, and Inline, bring the design language of Apple Watch complication to the iPhone; the complications of watchOS 9 are also driven by WidgetKit. The same widget code can cover the lock screen and watch face.

App Intents deal with another entry point problem. In the past, users would have to manually add shortcuts for functionality to appear in Siri or Shortcuts. iOS 16 combines App Intents with App Shortcuts, so users can use Siri, Shortcuts, and Spotlight to find app capabilities without first configuring them.

The underlying framework allows heavier scenes to be implemented

Metal 3 focuses on gaming and professional graphics. Fast resource loading reduces the loading overhead from storage to GPU through Metal IO; offline shader compilation puts the GPU shader binary into the project construction phase to generate; Mesh Shading API puts geometry generation into a more optimized rendering pipeline; MetalFX upscaling uses lower resolution rendering, and then performs temporal antialiasing and upscaling.

WeatherKit, MapKit, and VisionKit show another direction: system-level services are exposed to applications. WeatherKit provides native Swift API and REST API; MapKit’s 3D City Experience, camera control, and realistic elevation help applications express more precise locations; VisionKit’s Live Text API and Data Scanner API bring photos, paused video frames, text from live cameras, QR codes, and barcodes into the application process.

---

Detailed Content

Swift 5.7: Concurrency, Regex, Generics, and Plugins (05:49)

Swift paragraph back to concurrency first. Apple says Swift Concurrency has been adopted by more than 40,000 apps in its first year of release and can now be deployed to operating systems released in the last three years. Async Algorithms putzip、throttleThis type of sequencing algorithm is brought to asynchronous sequences; distributed actors extend the actor isolation model between processes, devices, or Swift on Server backends.

// Swift entry point explicitly mentioned in the transcript
async/await
for
try/catch
zip
throttle
distributed actor

Key points:

• async/awaitKeep asynchronous sequence code reading linearly. -forUsed on async sequence, continuing the writing method of ordinary sequence. -try/catchHandling failures in network streaming data. -zipis an example of a combinatorial algorithm provided by Async Algorithms for asynchronous sequences. -throttleBuilt on new clock types to slow down sequence update frequency. -distributed actorMark actors and methods that can be accessed remotely.

Regex paragraphs solve string parsing problems. The Food Truck order in the speech is data packaged into strings. The new regex literal can be checked by the Swift compiler and the matching results are displayed in the playground; the regex builder converts complex patterns into Swift code and can also be used.hexDigitThis type of CharacterClass expresses intent.

// Regex entry point explicitly mentioned in the transcript
regex literal
regex builder
.hexDigit
first match

Key points:

• regex literalMake regularity a language-level syntax, supporting highlighting and compiler checking. -regex builderWrite complex regular expressions into Swift code that is easier to change. -.hexDigitUse CharacterClass to express hexadecimal characters. -first matchIt is the operation of extracting the result from the order string in the demonstration.

Generic syntax has also been simplified.someCan express the parameter “a certain collection of songs”;anyIt can express values ​​that “can accommodate different collection types”. For example, the playlist array contains both song set and song array.

some
any

Key points:

• someUsed in parameter positions to reduce boilerplate code for generic function signatures. -anyUsed to save more dynamic existential values.

Swift Package plugins extend the build process into Swift Packages. Command plug-ins can be run on demand, and build plug-ins run when the project is built; they can be called from the command line or Xcode and will also request permissions in the sandbox. In the Food Truck example, a command plug-in calls SwiftFormat sorting import, another SwiftLint-based build plug-in prompts for the location of missing documentation, and the build plug-in goes to Xcode Cloud.

// Plugin form explicitly mentioned in the transcript
command plugin
build plugin
SwiftFormat
SwiftLint
Xcode Cloud

Key points:

• Command plugin is suitable for on-demand actions such as formatting, generation, and inspection.
• build plugin is suitable for rules that need to be executed for every build.
• SwiftFormat and SwiftLint are real plug-in uses given in the talk.
• Xcode Cloud will execute the build plug-in to make local and cloud rules consistent.

SwiftUI, Swift Charts, and cross-platform interfaces (18:36)

SwiftUI’s 2022 updates revolve around three pain points: navigation state, complex layouts, and progressive adoption. The new navigation API supports saving and restoring selections, replacing the entire navigation stack, and adapting to launch state, size class changes, and deep links. The Grid API is responsible for aligning multiple rows and columns, and the custom layout API can reuse custom layout logic.

// SwiftUI entry point explicitly mentioned in the transcript
new navigation API
Grid API
custom layout API
half sheets
Share Sheets
Transferable
UICollectionView

Key points:

• The new navigation API is used to express the navigation style that is most suitable for the application.
• Grid API handles multi-row and multi-column alignment.
• The custom layout API allows layout logic such as flow layout and radial layout to be reused.
• Half sheets provide secondary views that slide out from above the main view.
• Share Sheets get type-safe data transfer via Transferable.
• UIKit’s collection view cell can host SwiftUI views and supports swipe actions and cell backgrounds.

Swift Charts is the new framework for this talk. It is built on SwiftUI, uses declarative syntax to create charts such as line charts, bar charts, heat maps, stream graphs, etc., and comes with a customizable VoiceOver experience. In the Food Truck example, the developer uses several modifiers to change the color, change the position of the bar, add annotation, smooth the curve, and interact directly in Previews.

// Swift Charts capability explicitly mentioned in the transcript
Swift Charts
line chart
bar chart
heat map
stream graph
annotation
catmullRom

Key points:

• Swift Charts and SwiftUI use similar declarative expressions.
• Line chart and bar chart are the chart types used directly in the demonstration.
• Heat map and stream graph are more complex chart types supported by the framework.
• annotation is used to mark notes on charts.
• catmullRom is the interpolation strategy chosen for the curve in the demo.

WidgetKit, Live Activities and System Surfaces (31:28)

The lock screen widget reuses the design language of Apple Watch complication. Circular displays a small image, gauge, or a small amount of text; Rectangular provides a larger information canvas; Inline uses a small amount of text and SF Symbols to place above the iPhone clock. The complications of watchOS 9 are also driven by WidgetKit, so the same code can generate iPhone lock screen information and Apple Watch dial information.

// WidgetKit family and modifier entry point explicitly mentioned in the transcript
Circular
Rectangular
Inline
Supported Families
Gauge
VStack
widgetAccentable
rendering mode
environment property

Key points:

• Circular is suitable for progress, icons or short text.
• Rectangular is suitable for more information such as weather forecasts and order progress.
• Inline is suitable for very short text and SF Symbols.
• Supported Families declares the families supported by the widget.
• Gauge is a component of Food Truck that displays order progress. -widgetAccentableAdapt text and symbols on the lock screen to system tint.
• rendering mode and environment property are used to distinguish situations such as full-color rendering of the dial.

Live Activities handle real-time events. Activities such as scores, taxi progress, and fitness progress can be displayed directly on the lock screen and created using WidgetKit. The difference between it and ordinary widgets is that presentation and state are updated in real time, and SwiftUI can also animate between state changes. The presentation also stated that Live Activities will be available in subsequent updates to iOS 16.

WidgetKit
Live Activities
SwiftUI animation
state updates

Key points:

• WidgetKit is the entry point for creating Live Activities.
• Live Activities for real-time activities that users are following.
• SwiftUI animation supports visual transitions between states.
• state updates ensure that users can get the current information at a glance.

The Messages Collaboration API brings existing collaboration capabilities into Messages and FaceTime. When a user shares a link, the application can mark the link as collaborative, the system provides the required identifiers, and the recipient can gain access immediately after clicking on it. Users can initiate collaboration from a Share Sheet or drag & drop, and can also send content update notifications back to a Messages session.

// Collaboration entry point explicitly mentioned in the transcript
Messages Collaboration API
Share Sheet
drag & drop
collaboration popover

Key points:

• The Messages Collaboration API does not replace the collaboration logic in the application, but connects the conversation entry.
• Share Sheet puts the collaboration entrance in a location familiar to users.
• Drag & drop supports dragging collaboration content into Messages.
• collaboration popover lets users return to Messages or FaceTime conversations from the app.

App Intents vs. Passkeys: Discoverability and Login Security (38:59)

App Intents bring app functionality into Siri, Shortcuts, and Spotlight. The Food Truck in the speech wanted to expose “View the best-selling donuts today or this week”, so it imported App Intents and SwiftUI, and definedAppIntentstruct, for the time range enum incrementAppEnum,accomplishperform, and then define app shortcut phrase.

import AppIntents
import SwiftUI

AppIntent
AppEnum
perform
App Shortcut phrase
Siri tip

Key points:

• import AppIntentsandimport SwiftUIis the first step after creating a new Swift file in the demo. -AppIntentRepresents something the user can do in the application. -AppEnumGive parameter values ​​human-readable names, such as today and this week. -performExecute the intent and return a SwiftUI chart view, dialogue, or output value.
• App Shortcut phrase must contain the App name variable and can also contain parameters.
• Siri tips are placed in the application interface to help users know what to say.

Passkeys solves the three core problems of passwords: phishing, reuse, and server leakage. When creating a passkey, the device generates a unique key for a specific website or app and protects it with Face ID or Touch ID; the server only saves the public key. Users can also scan a QR code with their iPhone to log into the same Food Truck website on a friend’s PC.

// Passkey integration steps explicitly mentioned in the transcript
store public keys
issue authentication challenges
create a new passkey
sign in with it
WebAuthn

Key points:

• The backend needs to save public keys.
• The backend needs to issue authentication challenges.
• Apps and websites must provide access to create passkeys.
• The login process must support passkey sign in.
• WebAuthn is the open standard that passkeys are based on.

Metal 3, WeatherKit, VisionKit: system capabilities for more scenes (52:29)

The focus of Metal 3 is on reducing game loading and rendering bottlenecks. Metal IO makes the path of resources from high-speed SSD to unified memory to GPU more stable; offline shader compilation generates GPU shader binaries during the build phase, reducing runtime compilation; Mesh Shading API uses Object shader to determine the number of meshes, and then Mesh shader generates geometry; MetalFX upscaling uses lower resolution rendering, and then performs temporal antialiasing and upscaling.

// Metal 3 entry point explicitly mentioned in the transcript
Metal IO API
offline shader compilation
Mesh Shading API
Object shader
Mesh shader
MetalFX upscaling
temporal antialiasing

Key points:

• Metal IO API for fast resource loading.
• offline shader compilation advances shader binary generation to build time.
• Mesh Shading API avoids wrapping geometry generation results back into device memory.
• Object shader determines how many meshes to generate.
• The Mesh shader generates the actual geometry and sends it to the rasterizer.
• MetalFX upscaling renders at a low resolution first, then upscales to the target resolution.

WeatherKit provides native Swift API and REST API. In the Food Truck example, the application first adds the WeatherKit capability in Xcode and then calls it through Swift ConcurrencyWeatherServiceofweather(for:), pass in the location, and then take condition, prescription, and cloud cover to recommend a sunny location for the parking spot. Apple also stated that developer program membership includes 500,000 times per monthweather(for:location) API calls。

WeatherService
weather(for:)
location
condition
precipitation
cloud cover
weather(for:location)

Key points:

• WeatherService is the service entry for requesting weather in the demo. -weather(for:)Receives a location.
• Condition, prescription, and cloud cover are the data used by Food Truck to determine the parking point. -weather(for:location)is the API call name mentioned by Apple in the usage instructions.

VisionKit brings Live Text API and Data Scanner API. The Live Text API analyzes text and QR codes in images and paused video frames and provides quick actions; the Data Scanner API analyzes real-time camera input and is suitable for QR codes, barcodes, warehouse picking, distribution and POS scenarios. In the demo, Food Truck addsImageInteraction, and then adjust the button position through custom insets; during the order process, DataScanner is instantiated to find text, QR codes or barcodes.

ImageInteraction
Live Text API
Data Scanner API
DataScanner
text
QR codes
barcodes
custom insets
button configuration

Key points:

• ImageInteraction adds Live Text buttons and quick actions to the image view.
• Custom insets are used to avoid existing buttons in the application.
• button configuration adjusts the button style to match the application UI.
• DataScanner for live camera input.
• text, QR codes, and barcodes are the content types that DataScanner looks for in the demo.

---

Core Takeaways

• Connect CI/CD to Xcode Cloud

  • What: Build automated build processes from commit to TestFlight for iOS, iPadOS, macOS, watchOS projects.
  • Why it’s worth doing: This talk shows that Xcode Cloud is available and connects TestFlight, App Store Connect, Git provider, and REST API.
  • How ​​to start: First connect the existing Git repository to Xcode Cloud, and then use the REST API to connect to the team’s existing notification or release process.

• Make high-frequency status into lock screen widget and Apple Watch complication

  • What: Put order progress, fitness goals, inventory alerts, schedule countdowns into Circular, Rectangular or Inline widgets.
  • Why it’s worth doing: WidgetKit reuses the same model on iOS 16 lock screen and watchOS 9 complication, and platform differences are handled by the system.
  • How ​​to start: Declare the new family in Supported Families, using Gauge, VStack, system fonts andwidgetAccentableAdapt to lock screen and watch face.

• Expose in-app actions to Siri and Spotlight

  • What to do: Make actions such as “View this week’s sales trends”, “Record a glass of water” and “Start Pomodoro” available to be called directly by Siri and also appear in Spotlight searches.
  • Why it’s worth it: App Intents and App Shortcuts allow users to discover app capabilities without having to manually configure shortcuts first.
  • How ​​to start: Use AppIntent to describe the action, use AppEnum to describe the parameters, add perform, and then add Siri tip to the interface.

• Plan Passkeys migration for account system

  • What to do: Add passkey registration and login next to the existing username and password login.
  • Why it’s worth doing: Passkeys uses public key cryptography, and the server only saves the public key, which can reduce the risk of password reuse, phishing, and leakage.
  • How ​​to start: The backend first supports storing public keys and issuing authentication challenges, and then the client provides the entrance to create passkey and sign in with passkey.

• Improving offline processes with WeatherKit and VisionKit

  • What to do: Add weather decision-making, image text extraction and QR code scanning for distribution, inspection, travel and activity apps.
  • Why it’s worth it: WeatherKit provides Swift and REST APIs; VisionKit’s Live Text API and Data Scanner API can put system-level identification capabilities into custom processes.
  • How ​​to start: First add WeatherKit capability to the weather scene and call WeatherService; add ImageInteraction to the picture scene; use DataScanner to process text, QR codes or barcodes in the real-time code scanning scene.

---

Related Sessions

• What’s new in SwiftUI — An in-depth explanation of the new navigation, layout, Share Sheet, and cross-platform SwiftUI capabilities mentioned in this overview.
• Dive into App Intents — Continue to expand AppIntent, AppEnum, App Shortcuts, Siri phrase, and Spotlight discovery mechanisms.
• Discover Metal 3 — Detailed introduction to Metal IO, offline shader compilation, Mesh Shading API and MetalFX upscaling.
• Meet WeatherKit — Focus on WeatherKit’s Swift API, REST API, usage quotas, and privacy model.