WWDC Quick Look đź’“ By SwiftGGTeam
中文 EN 日本語
Build intelligent Siri experiences with App Schemas

Build intelligent Siri experiences with App Schemas

App IntentsApp SchemasApple IntelligenceSiriIndexedEntity
Watch original video

Highlight

In iOS 27, Siri gains the ability to understand app content, perform cross-app actions, and perceive screen context through App Schemas. Developers only need to model existing data as AppEntity, adopt system-defined schemas, and annotate views. Siri can then search, act on, and move app content using natural language.

Core Content

In the past, letting Siri control an app meant developers had to handle natural-language parsing themselves, write many custom intents, and make users remember fixed phrases. The experience was rigid, and the coverage was limited.

This year, Apple Intelligence turns Siri into an assistant that can truly “understand” apps. It no longer treats an app as a black box. Instead, it directly understands what entities inside the app are, what actions they support, and what is currently on screen. Developers no longer need to handle language understanding themselves; they just need to describe the app’s data structures and capabilities clearly. Siri translates the user’s words into concrete operations.

The core of this shift is App Schemas. App Schemas are system-defined sets of concepts, such as messages, contacts, and documents. When a developer’s AppEntity conforms to a schema, Siri already knows what the concept means, which properties it has, and which operations are commonly used with it. It does not need to learn everything from scratch.

For example, the user says, “Send Glow a message asking what movies they recommend.” Siri knows that “send a message” maps to the sendMessage schema in the messages domain, that “Glow” is a contact entity, and that “what movies they recommend” is the message content. It automatically resolves entities, fills parameters, and executes the intent. The app is only responsible for actually sending the message.

The more powerful part is cross-app actions. The user can say, “Send this email reply to my wife.” Siri first understands which message entity “this reply” refers to on screen, then exports it into a system-understandable format and passes it to the Mail app to send. The full flow involves screen awareness, content export, and cross-app transfer. Developers only need to annotate views and implement the Transferable protocol.

Siri’s three new capabilities

Understanding app content (01:57): Siri can now directly access app entities. When the user asks, “When and where is my next meeting?”, Siri can identify the “meeting” entity type, find the most relevant one, and return its time and location properties.

Performing app actions (02:19): When the user says, “Send the latest report to Mary,” Siri parses the “send” action, the “latest report” entity, and the “Mary” recipient, then calls the app’s sendMessage intent.

Perceiving screen context (02:44): When the user points at the screen and says, “Explain this paragraph” or “Look up reviews for this product,” Siri uses view annotations to know what content is on screen and resolves “this paragraph” or “this product” into concrete entities.

Semantic search vs. string queries

For most scenarios, IndexedEntity is the first choice. It indexes entity properties into the system semantic index, so Siri can match by meaning instead of matching exact text. If the user says, “What Flare said last time about movies,” Siri can find a message that mentioned films even if the exact word “movies” does not appear.

But IndexedEntity is not right for every case. For large datasets, server-side data, or data that changes frequently, EntityStringQuery is a better fit. Siri passes the user’s input string to the app, and the app decides how to search and match. The tradeoff is that you lose semantic understanding, but you gain full control.

Two APIs for screen awareness

One case is when the screen has one primary piece of content, such as reading a document or composing an email. Use NSUserActivity to mark the current activity, and Siri knows what “this” refers to.

The other case is when the screen shows multiple content items at the same time, such as a chat list or product list. Use the .appEntityIdentifier modifier to bind each list item to its entity identifier. Siri can distinguish “this message” from “that message” and supports commands like “forward the last one.”

Two modes for content transfer

When content enters your app from another app, there are two ways to handle it. If the content corresponds to existing data in your app, use IntentValueQuery to resolve it into an existing entity. If the content should create new data, use Transferable’s importing closure to convert the incoming value into a new entity. Many apps use both, depending on the scenario.

Details

Modeling data with AppEntity

AppEntity describes existing data instead of replacing your data model. It answers three questions: what this thing is, how to identify it, and which properties matter.

Using UnicornChat as an example, the app has three entity types: Contact, Conversation, and Message. Each conforms to the corresponding App Schema:

@AppEntity(schema: .messages.message)
struct MessageEntity: IndexedEntity {

    // Tell Spotlight which properties participate in semantic search
    @Property(indexingKey: \.textContent)
    var body: AttributedString?
}

Key points:

  • @AppEntity(schema: .messages.message) binds MessageEntity to the message schema in the messages domain, so Siri immediately understands that this is a “message” concept
  • The IndexedEntity protocol lets the system build a semantic index for the body property
  • indexingKey specifies the property that participates in semantic search, so Siri can find content by meaning rather than literal matching

String query fallback

When data cannot be indexed, implement EntityStringQuery:

struct ContactQuery: EntityStringQuery {
    func entities(matching string: String) async throws -> [ContactEntity] {
        let predicate = #Predicate<Person> { person in
            person.name.localizedStandardContains(string)
        }
        let descriptor = FetchDescriptor<Person>(predicate: predicate)
        let matches = try modelContext.fetch(descriptor)
        return matches.map(\.entity)
    }
}

Key points:

  • The EntityStringQuery protocol requires an implementation of entities(matching:)
  • Siri passes the user’s natural-language input through as-is, and the app is responsible for finding matching entities
  • The example uses SwiftData’s FetchDescriptor and #Predicate for local querying
  • Returning [ContactEntity] lets Siri continue the next operation

Annotating screen content

Let Siri understand which entity each item in a list corresponds to:

List {
    ForEach(messages) { message in
        MessageRow(message: message)
            .appEntityIdentifier(
                EntityIdentifier(
                    for: MessageEntity.self,
                    identifier: message.id
                )
            )
    }
}

Key points:

  • .appEntityIdentifier is a SwiftUI view modifier that associates a UI element with an AppEntity
  • EntityIdentifier must specify the entity type and unique identifier
  • When the user says, “Edit this message,” Siri resolves the specific MessageEntity from the view annotation
  • Only list items currently rendered on screen are annotated; off-screen items in a LazyVStack are not visible to Siri

Exporting entities for other apps

Make your content usable by other apps:

extension ContactEntity: Transferable {

    static var transferRepresentation: some TransferRepresentation {
        IntentValueRepresentation(
            exporting: \.person
        )
    }
}

Key points:

  • The Transferable protocol allows entities to move between apps
  • IntentValueRepresentation defines the export format; here, ContactEntity is exported as the system-standard IntentPerson
  • After export, intents in other apps can receive this contact, such as “call this contact”

Importing content to create a new entity

Receive content from another app and create new data:

extension ContactEntity: Transferable {

    static var transferRepresentation: some TransferRepresentation {
        IntentValueRepresentation(
            exporting: \.person,
            importing: { intentPerson in
                let contact = Contact(importing: intentPerson)
                ContactManager.shared.contacts.append(contact)
                return contact.entity
            }
        )
    }
}

Key points:

  • The importing closure runs when external content is received
  • It converts the system-standard IntentPerson into the app’s internal Contact model
  • After adding it to ContactManager, it returns the corresponding ContactEntity
  • Pay attention to thread safety: if ContactManager is bound to MainActor, the importing closure must handle actor isolation correctly

Resolving incoming intent values

When content from another app should match existing data:

struct ContactEntityQuery: IntentValueQuery {

    func values(for input: [IntentPerson]) async throws -> [ContactEntity] {
        let names = input.map(\.displayName)
        let descriptor = FetchDescriptor<Contact>()
        let contacts = try model.mainContext.fetch(descriptor)
        let matches = contacts.filter { contact in
            names.contains(where: { name in
                contact.name.localizedStandardContains(name)
            })
        }
        return matches.map(\.entity)
    }
}

Key points:

  • The IntentValueQuery protocol converts incoming system values into app entities
  • The example receives [IntentPerson] and matches local contacts by displayName
  • It uses localizedStandardContains for localized fuzzy matching
  • It returns the resolved [ContactEntity] for the intent to use

Xcode Fix-It completes schemas

Xcode checks schema completeness. If you implement sendMessage but do not implement the related draftMessage, the compiler reports an error and generates completion code:

// draftMessage skeleton generated by Xcode Fix-It
@main
struct DraftMessageIntent: AppIntent {
    static var title: LocalizedStringResource = "Draft Message"

    @Parameter
    var recipient: ContactEntity?

    @Parameter
    var content: String?

    @MainActor
    func perform() async throws -> some IntentResult {
        // Open the message composition interface
        return .result()
    }
}

Key points:

  • Some Siri scenarios require multiple related schemas to work together for a complete experience
  • Xcode exposes missing schemas at compile time instead of silently failing at runtime
  • Fix-It generates a full intent definition, parameter declarations, and implementation skeleton
  • Developers only need to fill in app-specific business logic

Key Takeaways

Build a Notes app that Siri can “understand”

What to build: Let users search notes with natural language, such as “find last week’s note about the budget,” and let Siri return results from the semantic index.

Why it is worth building: IndexedEntity puts local note content into the system semantic index, so users do not need to remember note titles or tags.

How to start: Wrap the Note model in an entity that conforms to AppEntity(schema: .notes.note), add indexingKey to content and title, and implement EntityStringQuery as a fallback for cloud-backed notes.

Build a to-do app that supports cross-app transfer

What to build: When the user sees a task in Mail and says, “Add this to my to-do list,” Siri parses the email content into a to-do item.

Why it is worth building: Transferable + importing lets content from other apps flow seamlessly into your app, so users do not need to copy and paste manually.

How to start: Make TodoEntity conform to Transferable, implement the importing closure for IntentValueRepresentation, and convert incoming text or URLs into Todo objects.

Build a screen-aware image editing app

What to build: When the user selects an image and says, “Apply a cinematic filter to this photo,” Siri knows that “this photo” refers to the selected image on screen.

Why it is worth building: .appEntityIdentifier lets Siri precisely identify on-screen content and supports natural references like “this” and “that.”

How to start: Add .appEntityIdentifier to each image view in the image grid, bind it to PhotoEntity, and implement an applyFilter AppIntent.

Build a schema-complete email client

What to build: Fully adopt the messages domain and support the full set of Siri actions, including sending, drafting, and searching.

Why it is worth building: Xcode guides you through completing all schemas in the domain. Once implemented, Siri’s operations on your email client feel consistent with the system Mail app.

How to start: Start with the sendMessage schema. Xcode Fix-It will tell you which related schemas are missing, and you can implement them one by one.

Comments

GitHub Issues · utterances