WWDC22 Day 4 recap

Highlight

Day 4 focused on the release of SwiftUI iPad interface organization, SwiftUI and UIKit interoperability, Vision framework optical flow calculation, and SharePlay experience design.

Core Content

SwiftUI iPad interface organization

Day 4 started with Raj explaining how to use SwiftUI to organize the iPad interface. Multi-window support in iPadOS 16 brings state restoration, deep linking, and richer programmatic control needs. (00:10)

SwiftUI’s navigation state is now serializable, supporting restoring UI state from a URL. This means that users can return to the correct interface location after closing the app and reopening it, or entering the app through a link.

SwiftUI integrates into UIKit

Sara shows how to use SwiftUI in a UIKit project. passUIHostingController, SwiftUI views can be embedded anywhere a View Controller is used. (00:21)

This incremental migration strategy allows existing UIKit projects to gradually introduce SwiftUI without requiring a one-time rewrite. You can start with a standalone page and gradually expand to more modules.

Vision framework optical flow calculation

Brett introduces the latest updates to the Vision framework. One demonstration involved calculating optical flow—tracking the direction and speed of motion for each pixel in a video. An example is a short video of a dog holding a water bottle on the beach. (00:30)

Optical flow computing is of great value for video analysis, action recognition, and AR scene understanding.

SharePlay Experience Design

SharePlay has been described as a “portal” that brings people into the same space via their mobile phones. Day 4’s session explained how to design an engaging SharePlay experience that makes users feel like they are really together with each other. (00:38)

Interface copywriting

Apple’s interface copywriting team shares writing principles: Interface writing starts with curiosity about the user on the other side of the screen. Good copywriting is part of the function, not just a visual embellishment. (00:48)

Detailed Content

Day 4 Technical Topic Classification

SwiftUI Advanced:

• iPad interface organization: multi-window, state restoration, deep linking
• UIKit interop: a progressive migration strategy
• Navigation state management: serialization and recovery

Computer Vision:

• Optical flow calculation: pixel-level motion tracking
• Vision framework new API

Social & Collaborative:

• SharePlay design pattern
• Group Activities framework

Design and Copywriting:

• Principles of interface copywriting
• User-centered design language

Practical path for interoperability between SwiftUI and UIKit

Typical path for incremental migration:

1. Independent page: Use firstUIHostingControllerMake a new page into SwiftUI
2. Component replacement: Gradually replace custom views in UIKit with SwiftUI views
3. Data layer unification: Use ObservableObject or @Observable to unify the data flow
4. Complete module: When a functional module is completely implemented in SwiftUI, remove the corresponding UIKit code

This path has the lowest risk and allows teams to learn SwiftUI by doing while maintaining product stability.

Core Takeaways

• Progressive SwiftUI Migration: Existing UIKit projects do not need to wait for a rewrite and can introduce SwiftUI starting from a settings page or new feature module. useUIHostingControllerWraps a SwiftUI view and uses it like a normal View Controller. The entry API isUIHostingController(rootView:)。

• Video Action Analysis Application: Use Vision’s optical flow calculation to analyze action trajectories in sports videos. It can be used for sports training (analyzing shooting postures), pet behavior analysis, or security monitoring. The entry API isVNGenerateOpticalFlowRequest。

• Multiple people synchronous viewing application: Use SharePlay and Group Activities framework to build a multi-person synchronous viewing platform. Users can invite friends to watch movies together, the playback progress is automatically synchronized, and real-time voice chat is also possible. The entry API isGroupActivityandGroupSession。

• iPad multi-window document application: Use SwiftUI’s multi-window support to make an iPad application that supports parallel editing of multiple documents. Each document has an independent window and supports opening specific documents directly from external links. The entry API isWindowGroupandonOpenURL。

Related Sessions

• SwiftUI on iPad: Organize your interface — Build an iPad interface with SwiftUI
• Bring SwiftUI to UIKit — Integrate SwiftUI in UIKit
• What’s new in Vision — Vision framework new features
• Design compelling SharePlay experiences — SharePlay Experience Design