WWDC Quick Look đź’“ By SwiftGGTeam
中文 EN 日本語
Explore prompt design & safety for on-device foundation models

Explore prompt design & safety for on-device foundation models

DesignMachine Learning
Watch original video

Highlight

This session is taught by HCI researcher Mary Beth and AI safety engineer Sprite. The first half covers prompt design tips for the on-device model (about 3 billion parameters); the second half lays out AI safety strategy with the “Swiss cheese” layered model.

Core Content

Many developers reach for the Foundation Models framework and treat it as a mini ChatGPT. Then strange things start happening: ask it to act as a calculator, the math is wrong; ask it to write a sort algorithm, the code barely runs; ask it about recent news, and it makes things up with a straight face. Rewriting the prompt does not help. The issue is not a bad prompt. The model has different limits.

Apple makes this clear in the session. The on-device model has about 3 billion parameters. It ships in iOS, iPadOS, macOS, and visionOS, and Apple Intelligence’s Writing Tools runs on top of it (01:48). Compared with server-side models that have hundreds of billions of parameters, this one is shrunk to fit in your pocket. It is good at summarization, classification, multi-turn dialog, text creation, text rewriting, and tag generation. It is bad at complex reasoning, math, code generation, and serving facts. World knowledge is limited and hallucinations happen, so “treat it as a fact database” is a dead end from day one.

The safety section in the second half is the core of the session. The Foundation Models framework ships with a guardrail that Apple trained, and it runs on both input and output sides — even if a clever prompt slips past the input filter, the output side still blocks harmful content (12:30). But the framework’s safety is only the foundation. Apple repeats this point: developers must stack their own safety layers on top. Sprite uses Swiss cheese to picture layered defense — every slice has holes, but the chance of all holes lining up at once is very low (17:53). This mental model matters more than any specific trick.

Detailed Content

Model Capability Boundaries

The trade-offs of the on-device model boil down to a few hard rules (03:32):

  • It is bad at complex reasoning. Break the task into smaller steps before feeding it.
  • Hand math tasks to non-AI code. It is more reliable.
  • Code generation is not its strength. Do not use it as Copilot.
  • For factual questions, do not rely on it unless you have verified the model knows that specific topic.

Mary Beth gives the example “list 10 popular bagel flavors” (04:25). The model knows bagels, but describes a plain bagel as “with a bunch of toppings” — that is a hallucination. She then flips the framing: in a bakery mini-game, when the model generates customer order dialog, an occasional slip-up turns into a punchline. Whether you can tolerate hallucinations depends on the scenario.

Prompt Design Techniques

Start with the simplest Swift code (01:11):

let session = LanguageModelSession()
let prompt = "Generate a bedtime story about a fox."
let response = try await session.respond(to: prompt)

Key points:

  • LanguageModelSession() creates one session, which holds an ongoing conversation context.
  • prompt is the natural-language instruction to the model. You can write it in any language Apple Intelligence supports.
  • respond(to:) is an async throwing method that returns the model’s generated text.

Several concrete tips for tuning prompts (06:18):

  • Use “in three sentences” or “in a few words” to shorten output; use “in detail” to lengthen it.
  • Use a role to control style, e.g. “You are a fox who speaks Shakespearean English”.
  • Write prompts as clear commands. One task per prompt, specific enough.
  • Embedding fewer than 5 examples in the prompt can lift task performance noticeably.
  • To block a kind of output, use the all-caps command “DO NOT”. The effect is close to talking to it sternly (08:01).

The best place to debug is Xcode’s #Playground. Edit one line of the prompt and the right-side canvas updates instantly (08:11).

Instructions vs. Prompt

Instructions are another kind of prompt. You pass them in when you create the session, telling the model “respond to every prompt that follows by these rules” (08:51):

let session = LanguageModelSession(
    instructions: """
    You are a helpful assistant who generates scary stories \
    appropriate for teenagers.
    """
)
let response = try await session.respond(to: "Write a poem about bagels.")

Key points:

  • instructions take effect before all user prompts. They define the model’s persona and behavior limits.
  • The model is trained to obey instructions first, so safety policy is most stable when written here.
  • Even if the user prompt is “write a poem about bagels”, the whole reply keeps the assigned tone (scary, in this case).
  • Never put any user input into instructions. Only the developer writes instructions.

Guardrail Error Handling

When you call the model, catch the safety error the guardrail throws (12:56):

do {
    let response = try await session.respond(to: prompt)
    // Use the response
} catch LanguageModelSession.GenerationError.guardrailViolation {
    // The input or output was blocked by the guardrail
} catch {
    // Other errors
}

Key points:

  • The guardrail runs on both input and output. If the input slips through, the output still gets a second check.
  • For proactive scenarios (e.g. the app generating content on its own), you can swallow the error silently and not interrupt the UI.
  • For user-initiated scenarios, give clear feedback — an alert, or, like Image Playground, an alternative action that undoes the last prompt (13:43).

Three Modes for Handling User Input

When you wire user input into the model, there are three safety levels (15:26):

  1. Fully open: user input becomes the prompt directly. Most flexible, highest safety risk.
  2. Hybrid: the developer writes a prompt template and slots the user input into a fixed spot.
  3. Fully controlled: the app offers a fixed set of built-in prompts for the user to pick from. Safest, least flexible.

Control and flexibility are inversely related. Pick the level by use case. If mode 3 works, do not jump to mode 1.

Use-Case Specific Mitigation

The guardrail can only catch “universally harmful” content. It cannot read your business context. Sprite gives two concrete examples (16:39):

  • When generating bagel flavors, the model may include allergens like nuts or garlic. Fix this by adding an allergy warning in the UI, or collecting dietary restrictions in settings and filtering on them.
  • A trivia generator should avoid controversial or sensitive topics. You can add rules to the instructions, keep a keyword blacklist, or train a classifier for more robust filtering.

Evaluation and Testing

The last step is evaluation. Build two datasets: one covers normal use cases, the other collects prompts likely to trigger safety issues (18:44). Write a command-line tool or a UI test app to run end-to-end checks. For small datasets, review by hand; for large ones, use another LLM as the judge. Also test the unhappy path — when the guardrail fires, does the app behave as you expect?

Key Takeaways

1. Add an “input tier” to your app’s design

Why bother: Foundation Models is free, local, and privacy-safe on-device capability, perfect for lightweight natural-language interaction. But prompt injection risk is real. Decide on the input tier before you build.

How to start: Pick a tier at the entry point. If preset buttons (fully controlled) work, skip the free-form text box. If free-form input is required, wrap the user text inside your own prompt template. Never splice user text into instructions.

2. Treat guardrail errors as first-class citizens

Why bother: Many developers brush off guardrail errors as generic errors, and the user ends up staring at a dead-end alert. That is a fast way to lose trust.

How to start: Split proactive and user-initiated scenarios. Proactive ones skip silently. User-initiated ones get an actionable next step — undo the input, swap to an example, fall back to a non-AI path. Look at Image Playground’s “undo prompt” design.

3. Add a “negative-emotion fallback” paragraph in your instructions

Why bother: When users journal, chat, or brainstorm in your app, negative content shows up. The default model reply may sound cold or pour fuel on the fire, which hurts how users feel about the app.

How to start: Add a line to instructions like “Respond to negative prompts in an empathetic and wholesome way.” Then list topics the app does not discuss (medical diagnosis, legal advice, self-harm, etc.).

4. Build a minimum viable pipeline for prompt evaluation

Why bother: The model and the guardrail keep updating. A prompt that works today may drift tomorrow. Without an evaluation pipeline, regressions only surface through user complaints.

How to start: Make two JSON datasets (normal cases + risky cases). Write a Swift command-line runner that prints each prompt’s response. Label small sets by hand; for big sets, use a larger LLM as the judge. Run it before and after every prompt change or SDK upgrade.

5. Replace “format-by-prompt” with Guided Generation

Why bother: Asking the model to “output a JSON array” is common but brittle. Hallucinations and format errors are common. Guided Generation directly constrains the output to a given struct, enum, or array, killing format issues at the source.

How to start: Convert every “please output in the following JSON format” prompt to a @Generable Swift type. Migrate one hot path first, confirm it is reliable, then move the rest.

6. Build a prompt lab in Xcode #Playground

Why bother: Prompt tuning is high-frequency and low-coupling work. Traditional unit tests give feedback too slowly. #Playground shows the result the moment you change a line — exactly right for prompts, which are observable but hard to formalize.

How to start: Add PromptLab.swift to the project. Copy each production prompt in and run parameterized experiments across four buckets: normal input, long input, empty input, malicious input. Once a change is stable, merge it back to the main code.

Comments

GitHub Issues · utterances