WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
WWDC26 Platforms State of the Union Recap

WWDC26 Platforms State of the Union Recap

Agentic CodingCore AIFoundation ModelsXcode 27SwiftUI
Watch original video

Highlight

The 5-minute highlight version. Apple Intelligence has been rebuilt underneath, Xcode 27 embraces agentic coding, and SwiftUI fills long-standing interaction gaps. For the full story, see Platforms State of the Union.

Core Ideas

Apple Intelligence rebuilt from the ground up

Apple worked with Google to rebuild Apple Foundation Models on Gemini technology. For developers, three changes matter most:

  • Foundation Models framework adds image input and cloud model support, letting complex tasks switch smoothly to third-party cloud models such as Claude and Gemini.
  • Dynamic Profiles let you dynamically switch models, tools, and instructions within the same session, dramatically reducing the glue code needed to build AI agents.
  • Core AI is a new on-device model runtime framework with memory-safe Swift APIs, fine-grained quantization, and custom GPU kernels. It can handle everything from real-time vision models on iPhone to multi-billion-parameter LLM agents on Mac.

App Intents connect apps to system AI

App Intents help Siri understand what your app can do. The new View Annotations API even lets a user point at something on screen and say “send this to Kevin”; the system can locate the exact content and perform the action. Your app is no longer an island. It becomes a node in the system intelligence network.

SwiftUI is faster and more flexible

  • Drag to reorder and swipe actions now work in any container, not just List.
  • Nested layouts resize twice as fast.
  • AsyncImage now caches automatically.
  • Toolbars add visibilityPriority and overflowMenu, so important buttons remain visible as a window shrinks.
  • iOS apps on iPad and Mac support free resizing.

Xcode 27: the full agentic coding workflow

  • 30% smaller, Apple Silicon only, with settings synced through iCloud.
  • Device Hub replaces Simulator, unifying virtual and physical device management with pinch to zoom, live resizing, and remote control of real devices.
  • Agentic Coding integrates Anthropic, OpenAI, and Google models. AI conversations can be split and stacked in the Navigator like regular files.
  • In Plan mode, the agent proposes a plan before writing code, and can directly drive a running app to run tap, scroll, and swipe tests.
  • Built-in experts cover SwiftUI, accessibility, sizing, testing, performance, and more, and agents can call them directly.
  • Plugin extensions are supported, with official Figma and GitHub plugins installable in one click.

Design: Liquid Glass unifies the system style

macOS window corner radii are tightened into a consistent system, app icons gain sharper rendering automatically, and Icon Composer adds refraction effects. iOS apps can be freely resized on iPad and Mac, while the new resizable iOS simulator and previews make cross-size testing easier.

Details

1. Core AI: a new model-runtime paradigm for on-device AI

Core AI is Apple’s biggest on-device AI infrastructure change this year. It is a new model inference framework independent of Core ML, supporting deployment across the full spectrum from tiny vision models to multi-billion-parameter LLMs. Key capabilities include:

  • Memory-safe Swift APIs: model loading, inference, and memory management are protected by Swift’s type system, with no need to manually handle raw buffers.
  • Fine-grained quantization: 4-bit and 8-bit quantization are supported, and developers can choose quantization strategies by layer or even by tensor to precisely balance accuracy and speed.
  • Custom GPU kernels: for specific operators, developers can write custom kernels in Metal and register them into Core AI’s graph. This is especially important for vision models and custom architectures.
  • A unified cross-device model: the same code runs on iPhone, iPad, and Mac. Core AI automatically chooses an execution path based on device capability, across ANE, GPU, and CPU.

2. Dynamic Profiles: orchestration infrastructure for AI agents

Dynamic Profiles in the Foundation Models framework address the pain of multi-model collaboration. A Profile defines a model, tool set, system instructions, and context-window strategy, and developers can switch profiles dynamically at runtime:

  • Seamless local/cloud switching: simple tasks use the on-device model, while complex reasoning can automatically fall back to cloud models such as Claude or Gemini. The switch is transparent to the app layer.
  • Multi-agent collaboration: different Profiles can call one another, such as a “planning agent” using a strong reasoning model to generate steps and an “execution agent” using a lightweight model to complete them one by one.
  • Context isolation: each Profile has its own context window, preventing tool-call history from polluting the main conversation flow.

3. Xcode 27 Agentic Coding: from Copilot to Autopilot

Xcode 27 upgrades AI integration from code completion into a complete “agent workflow”:

  • Plan mode: the agent analyzes the requirement, generates an execution plan, and waits for developer approval before writing code, reducing the feeling that AI is making uncontrolled changes.
  • App-driven testing: the agent can directly operate a running app with taps, scrolling, swipes, and text input, verify whether behavior matches expectations, and automatically fix failing cases.
  • Domain expert system: built-in experts for SwiftUI layout, accessibility, performance, and more can be called automatically when the agent encounters specialized problems, instead of relying on generic-model guessing.
  • Conversations as files: AI conversations live in the project as .ai files, so they can be tracked by git, reviewed, and shared like normal files.

4. SwiftUI fills in interaction gaps

This year’s SwiftUI updates focus on making complex interactions simple:

  • Universal Drag-to-Reorder: any container that supports ForEach, including Grid, HStack, and custom layouts, can support drag reordering without manually handling onMove and DropDelegate.
  • Universal Swipe Actions: views outside List can define swipe actions too, with support for custom animation curves and thresholds.
  • Nested layout performance doubles: improved layout caching makes deeply nested combinations of VStack, HStack, and ZStack recalculate about 2x faster during size changes.
  • Freely resizable iOS apps: iOS apps are no longer constrained to fixed aspect ratios on iPad and Mac, so developers need to reconsider responsive layout edge cases.

5. App Intents View Annotations: from “open the app” to “operate this exact UI”

The View Annotations API lets Siri and system AI precisely locate specific elements in an app interface:

  • Developers mark actionable views with the .intentAnnotation() modifier, and the system automatically extracts the element type, content, and location.
  • A user can say “send this picture to Kevin”; the system can identify which picture on the current screen is “this picture” and which App Intent corresponds to “send to Kevin.”
  • This requires App Intents definitions to be deeply connected to UI structure, raising the bar for the granularity of Intent design.

Key Takeaways

  1. The window for on-device AI apps is open: Core AI makes it feasible to run LLMs on iPhone. Combined with Apple’s privacy story, product ideas like a fully local personal assistant or offline intelligent photo organizer now have a technical foundation. Independent developers can find differentiated space here.

  2. Agentic Coding changes the development workflow: Xcode 27’s Plan mode plus app-driven testing means AI is no longer just a code-writing tool. It can become a collaborator that completes the loop from requirement understanding to implementation to verification. Teams need to rethink code review processes and ownership for AI-generated code.

  3. Multi-model orchestration becomes a default capability: Dynamic Profiles suggest that future apps will not bind themselves to one model. They will dynamically choose the best model for each task. Developers need a “models as services” mindset and should treat model switching as infrastructure, not a temporary hack.

  4. SwiftUI is filling in the last missing pieces: Universal drag reorder and swipe actions remove many cases where teams had to fall back to UIKit because SwiftUI could not do the job. New projects can choose SwiftUI as the only UI framework more confidently.

  5. App Intents are the new SEO: View Annotations greatly increase the chance that app features can be discovered and invoked by system AI. The better your Intent design, the more visible your app becomes in the Apple Intelligence ecosystem. This may become a new user acquisition channel.

Comments

GitHub Issues · utterances