WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Unwrap PaperKit

Unwrap PaperKit

PaperKitPencilKitCanvasUI Frameworks
Watch original video

Highlight

PaperKit is the canvas engine Apple uses internally in apps such as Notes, Preview, and Freeform. Starting in iOS, macOS, and visionOS 27, it is available to developers with a full canvas data model, element manipulation, and custom adornment layers.

Core Ideas

In the past, building an app canvas that supported drawing, markup, and draggable images meant handling touch events, drawing logic, undo and redo, gesture recognition, layer management, and much more. The implementation work was large, and the result was still hard to make feel consistent with Apple’s built-in apps.

PaperKit is Apple’s unified framework for solving that problem. It replaces simple drawing tools with a complete canvas experience where pencils, shapes, text, and images all work together. When you sketch in Notes, sign and highlight in Preview, or brainstorm in Freeform, PaperKit is behind the experience.

The framework has three main layers:

Data model layer: PaperMarkup exposes the ordered collection of all elements on the canvas through subelements, so you can read, create, and modify any element.

Element control layer: Every element conforms to the Markup protocol and has shared properties such as frame and rotation, plus type-specific properties. The new allowedInteractions API lets you precisely control how each element can be manipulated. Moving, resizing, rotating, deleting, changing style, and selecting can all be controlled independently.

Adornment layer: Add temporary UI controls above the canvas, such as buttons, notes, and collaboration markers. These controls do not become part of the document, are not saved or exported, and exist only during editing. They automatically follow zooming and scrolling.

Details

Data model and subelements

At the center of PaperKit is PaperMarkup, which represents the full contents of the canvas. ([01:36](https://developer.apple.com/videos/play/wwdc2026/372/?time=96))

import PaperKit

func generateMarkup(pageSize: CGSize, panelFrames: [CGRect], configuration: ShapeConfiguration) -> PaperMarkup {
    var markup = PaperMarkup(bounds: CGRect(origin: .zero, size: pageSize))
    var subelements: MarkupOrderedSet = markup.subelements
    for panelFrame: CGRect in panelFrames {
        let shape = ShapeMarkup(frame: panelFrame, configuration: configuration)
        subelements.append(shape)
    }
    markup.subelements = subelements
    return markup
}

Key points:

  • PaperMarkup needs bounds when initialized, which defines the canvas size.
  • subelements is a readable and writable ordered collection of type MarkupOrderedSet.
  • ShapeMarkup is a shape element. It needs a frame and configuration when initialized.
  • After changing subelements, assign it back to markup.subelements for the change to take effect.

Controlling element interactions

By default, elements on the canvas can be selected, dragged, and deleted freely. Some scenarios need template elements to be locked. ([03:03](https://developer.apple.com/videos/play/wwdc2026/372/?time=183))

import PaperKit

func generateMarkup(pageSize: CGSize, panelFrames: [CGRect], configuration: ShapeConfiguration) -> PaperMarkup {
    var markup = PaperMarkup(bounds: CGRect(origin: .zero, size: pageSize))
    var subelements: MarkupOrderedSet = markup.subelements
    for panelFrame: CGRect in panelFrames {
        var shape = ShapeMarkup(frame: panelFrame, configuration: configuration)
        shape.allowedInteractions = .readOnly
        subelements.append(shape)
    }
    markup.subelements = subelements
    return markup
}

Key points:

  • allowedInteractions is an option set of type MarkupInteractions.
  • .readOnly is a combined flag that disables moving, resizing, rotating, deleting, style changes, and selection all at once.
  • You can also control individual behaviors, such as allowing movement while disallowing deletion.

Style changes

All elements have type-specific properties. ShapeMarkup has strokeColor and fillColor. ([04:22](https://developer.apple.com/videos/play/wwdc2026/372/?time=262))

import PaperKit

func updatePanelColor(_ selectedColor: CGColor) {
    guard var markup: PaperMarkup = paperMarkupViewController.markup else { return }
    var subelements: MarkupOrderedSet = markup.subelements
    for element in subelements {
        guard var shape = element as? ShapeMarkup else { continue }
        shape.strokeColor = selectedColor
        shape.fillColor = selectedColor.copy(alpha: 0.15)
        subelements.updateOrAppend(shape)
    }
    markup.subelements = subelements
    markup.backgroundColor = selectedColor.copy(alpha: 0.15)
    paperMarkupViewController.markup = markup
}

Key points:

  • When iterating over subelements, cast each item because the collection contains different types that conform to Markup.
  • updateOrAppend updates an existing element or appends a new one.
  • After the update is complete, reassign the result to paperMarkupViewController.markup.

Adornments

Adornments are temporary controls overlaid on the canvas. They are not saved into the document, so they are a good fit for buttons, annotations, and collaboration UI. ([05:53](https://developer.apple.com/videos/play/wwdc2026/372/?time=353))

import PaperKit

func addPanelAdornments(for page: Page) {
    var adornments: [MarkupAdornment] = []
    for (panelIndex, panel) in page.panels.enumerated() {
        let adornmentID = UUID()
        adornmentPanelMapping[adornmentID] = panelIndex
        let center = CGPoint(x: panel.midX, y: panel.midY)
        let adornment = MarkupAdornment(
            id: adornmentID,
            anchor: .canvas(location: center),
            imageConfiguration: .systemImage("photo.badge.plus"),
            dragRegion: .fixed,
            scalesWithZoom: false
        )
        adornments.append(adornment)
    }
    paperMarkupViewController.adornments = adornments
}

Key points:

  • MarkupAdornment uses a UUID as a unique identifier for later click handling.
  • anchor uses .canvas(location:) to bind to canvas coordinates, so it follows zooming and scrolling automatically.
  • imageConfiguration supports SF Symbols.
  • dragRegion: .fixed means the button position is fixed, and scalesWithZoom: false means it does not grow with zoom.

Handling adornment taps

import ImagePlayground
import PaperKit

func paperMarkupViewController(_ paperMarkupViewController: PaperMarkupViewController, didTapAdornmentWithID id: UUID) {
    guard let panelIndex = adornmentPanelMapping[id] else { return }
    activeImageGenerationPanelIndex = panelIndex

    let imagePlaygroundViewController = ImagePlaygroundViewController()
    imagePlaygroundViewController.delegate = self
    present(imagePlaygroundViewController, animated: true)
}

Key points:

  • Implement the delegate method to receive adornment tap events.
  • Map the id back to a specific business object, such as a comic panel index.
  • Present another interface when needed, such as Image Playground.

Inserting the generated image

import ImagePlayground
import PaperKit

func imageViewController(_ imageViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL) {
    guard let panelFrame = activeGenerationPanelFrame,
          let paperMarkupViewController = pageViewController.paperViewController,
          var markup = paperMarkupViewController.markup,
          let image = UIImage(contentsOfFile: imageURL.path) else { return }

    let imageMarkup = ImageMarkup(frame: panelFrame, image: image)
    markup.subelements.append(imageMarkup)
    paperMarkupViewController.markup = markup
}

Key points:

  • ImageMarkup creates an image element and needs both a position and a UIImage.
  • Append it directly to subelements to insert it.
  • Updating markup refreshes the view.

Relationship with PencilKit

PaperKit is built on PencilKit and supports Apple Pencil drawing. Every stroke becomes a markup element and can use PencilKit’s model APIs. These APIs now support character recognition and Bezier curve path conversion.

Key Takeaways

Comic creation app: Build a comic editor like the demo. Use allowedInteractions to lock the panel layout, add a “Generate Image” button with adornments, and combine it with Image Playground so users can create quickly. Entry point: ShapeMarkup + MarkupAdornment.

Coloring book app: Provide prebuilt line-art templates as read-only content, letting users color only specified regions. Iterate through subelements to find specific shapes and modify fillColor.

Collaborative annotation tool: Use adornments to show collaborators’ presence, cursor positions, and comment markers. This information is not saved into the document but can update in real time. Entry point: MarkupAdornment + periodic refresh.

Handwritten note search: Use PencilKit’s character recognition APIs to convert handwritten content to text, then insert TextMarkup elements so notes become searchable.

Template filling tool: Create document templates such as invoices, contracts, and forms. Lock the template structure with readOnly, and let users fill only the blank areas.

Comments

GitHub Issues · utterances