WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Explore enhancements to your spatial business app

Explore enhancements to your spatial business app

App ServicesSpatial ComputingBusiness and Education
Watch original video

Highlight

visionOS 26 opens UVC video and Neural Engine access to every developer, and adds four new enterprise APIs: Window Follow Mode, shared coordinate space, content capture protection, and Camera Region.

Core Content

Last year visionOS first opened low-level capabilities to enterprise customers, but the cost was high. To grab UVC camera video, you had to apply for a managed entitlement and ship a license file. Even Neural Engine access ran through the enterprise pipeline. A small team that wanted to use a USB camera for remote diagnostics had to walk through the full enterprise developer compliance process just to get approved.

visionOS 26 loosens this up. Apple engineer Alex Powers announced in the session that UVC video access and Neural Engine access are open to all developers this year, with no enterprise license or entitlement required. Object tracking moves beyond clicking around in the CreateML app and now supports command-line training, so you can wire it straight into a CI pipeline. License management has moved into the Apple Developer account, and renewals push to the device over the air.

The remaining enterprise capabilities split into two tracks. On the user experience side, you get Window Follow Mode (windows that follow the user), SharedCoordinateSpaceProvider (multiple Vision Pros in the same room aligning their coordinate systems), and contentCaptureProtected (sensitive content masked automatically in screen recording and sharing). On the environment perception side, Camera Frame Provider can fetch the left and right cameras separately or as a stereo pair, and the new Camera Region API can lift a small patch of the real world out for zooming and contrast enhancement, which fits use cases like reading a remote instrument panel.

Detailed Content

Train an object tracking model from the command line (03:00)

xcrun createml objecttracker -s my.usdz -o my.referenceobject

Key points:

  • xcrun createml objecttracker: invokes the CreateML object tracking subcommand, which shares the same training parameters as the GUI CreateML app.
  • -s my.usdz: the input USDZ model file, used to generate the reference object.
  • -o my.referenceobject: the output asset path, ready to drop into an automated pipeline and batch over files.

Check license status with VisionEntitlementServices (04:28)

import VisionEntitlementServices

func checkLicenseStatus() {
    let license = EnterpriseLicenseDetails.shared

    guard license.licenseStatus == .valid else {
        print("Enterprise license is not valid: \(license.licenseStatus)")
        return
    }

    if license.isApproved(for: .mainCameraAccess) {
        print("Main Camera Access approved. Enabling feature...")
    } else {
        print("Main Camera Access not approved.")
    }
}

Key points:

  • EnterpriseLicenseDetails.shared: a singleton that wraps the current app’s license info, including status and expiration date.
  • licenseStatus == .valid: confirm the overall license is valid first, so you don’t run enterprise logic against an expired license.
  • isApproved(for: .mainCameraAccess): checks at entitlement granularity whether a given enterprise API is approved, so the app can decide at runtime whether to turn the feature on.
  • The framework also supports the Increased Performance Headroom entitlement check, so you can confirm the permission before running a heavy workload.

Custom shared spaces with SharedCoordinateSpaceProvider (10:04)

import ARKit

class SharedCoordinateSpaceModel {
    let arkitSession = ARKitSession()
    let sharedCoordinateSpace = SharedCoordinateSpaceProvider()
    let worldTracking = WorldTrackingProvider()

    func runARKitSession() async {
        do {
            try await arkitSession.run([sharedCoordinateSpace, worldTracking])
        } catch {
            reportError("Error: running session: \(error)")
        }
    }

    func pushCoordinateSpaceData(_ data: Data) {
        if let coordinateSpaceData = SharedCoordinateSpaceProvider.CoordinateSpaceData(data: data) {
            sharedCoordinateSpace.push(data: coordinateSpaceData)
        }
    }

    func pollCoordinateSpaceData() async {
        if let coordinateSpaceData = sharedCoordinateSpace.nextCoordinateSpaceData {
            // Send my coordinate space data
        }
    }

    func addWorldAnchor(at transform: simd_float4x4) async throws {
        let anchor = WorldAnchor(originFromAnchorTransform: transform, sharedWithNearbyParticipants: true)
        try await worldTracking.addAnchor(anchor)
    }
}

Key points:

  • SharedCoordinateSpaceProvider runs alongside WorldTrackingProvider on an ARKitSession, reusing the familiar ARKit data provider model.
  • nextCoordinateSpaceData: pulls the local alignment data the device wants to broadcast to other participants. The developer chooses the transport — TCP, UDP, or a custom protocol.
  • push(data:): feeds CoordinateSpaceData received from other devices back into ARKit to build a shared coordinate system.
  • WorldAnchor(... sharedWithNearbyParticipants: true): marks the anchor as shared, so other Vision Pros in the room see the same anchor at the same position.
  • The eventUpdates async sequence notifies the app when participants join or leave and when the share toggle changes (10:04).

Block sensitive content with contentCaptureProtected (12:50)

struct SecretDocumentView: View {
    var body: some View {
        VStack {
            Text("Secrets")
                .font(.largeTitle)
                .padding()

            SensitiveDataView()
                .contentCaptureProtected()
        }
        .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top)
    }
}

Key points:

  • contentCaptureProtected() is a SwiftUI view modifier and requires the protected content entitlement.
  • Whichever layer you apply it to is the layer that gets masked automatically in system screenshots, screen recording, Screen Mirroring, and SharePlay.
  • The user wearing the Vision Pro still sees the original content; outside viewers see a blur or block.
  • It also works around an entire RealityKit scene, not just 2D views.

Camera Region: lift a patch of reality (16:48 / 21:15)

import SwiftUI
import VisionKit

struct InspectorView: View {
    @Environment(CameraFeedDelivery.self) private var cameraFeedDelivery: CameraFeedDelivery

    var body: some View {
        CameraRegionView(isContrastAndVibrancyEnhancementEnabled: true) { result in
            var pixelBuffer: CVReadOnlyPixelBuffer?
            switch result {
            case .success(let value):
                pixelBuffer = value.pixelBuffer
            case .failure(let error):
                reportError("Failure: \(error.localizedDescription)")
                cameraFeedDelivery.stopFeed()
                return nil
            }

            cameraFeedDelivery.frameUpdate(pixelBuffer: pixelBuffer!)
            return nil
        }
    }
}

Key points:

  • CameraRegionView comes from VisionKit. It’s a high-level wrapper that treats the window as a viewfinder pointed at a patch of the real world.
  • isContrastAndVibrancyEnhancementEnabled: true: turns on contrast and color enhancement, which suits distant gauges and needle readings.
  • The closure receives a pixelBuffer you can hand to your own video pipeline for remote assistance, OCR, or live streaming.
  • The lower-level ARKit equivalent is CameraRegionAnchor(originFromAnchorTransform:width:height:cameraEnhancement:), which lets you specify modes like .stabilization and deliver per-frame pixel updates through cameraRegionProvider.anchorUpdates(forID:).
  • Apple recommends keeping a single CameraRegionAnchor to no more than one-sixth of the visible area; beyond that, resource cost climbs sharply.

Core Takeaways

  • Push USB cameras further: UVC video access opens to every developer this year, with no enterprise license required.

    • Why it matters: teams that were turned away by license requirements can now plug in industrial cameras, thermal imagers, microscopes, and other specialty hardware directly.
    • How to start: get a Vision Pro Developer Strap, follow the Building spatial experiences for business apps doc to wire up the UVC API, get a basic webcam video stream working first, and layer business logic on top.

  • Train object tracking models from the command line: xcrun createml objecttracker makes model training scriptable.

    • Why it matters: what used to be one USDZ per product SKU and one manual click can now be a shell script that processes dozens of objects, with referenceobjects produced automatically in CI.
    • How to start: read Implementing object tracking in your app, install Xcode command line tools in your CI container, and feed USDZ inputs and referenceobject outputs through your artifact pipeline.

  • Apply contentCaptureProtected() uniformly to sensitive views: one modifier controls screenshot, screen recording, and sharing behavior.

    • Why it matters: in finance, healthcare, and design-asset scenarios, the biggest compliance pain is “an employee accidentally turned on SharePlay or screen recording.” This API moves control into the SwiftUI view layer.
    • How to start: list every subview that shows sensitive data, wrap them in a single SensitiveContent component that applies .contentCaptureProtected() internally, and request the protected content entitlement in your enterprise developer account.

  • Multi-user Vision Pro collaboration over a local network: use SharedCoordinateSpaceProvider to run a shared coordinate space on your own network.

    • Why it matters: SharePlay over iCloud is too heavy, and closed-network sites like factories and hospitals need a fully local collaboration channel. The new ARKit API hands the network layer entirely to the developer.
    • How to start: run both SharedCoordinateSpaceProvider and WorldTrackingProvider in the same ARKitSession, exchange CoordinateSpaceData over Bonjour + UDP on the LAN in both directions, and add shared WorldAnchors on top.

Comments

GitHub Issues · utterances