Discover Metal 3

Highlight

Metal 3 introduces fast asset loading, offline compilation, MetalFX Upscaling, Mesh Shaders, optimized ray tracing and machine learning acceleration, with a new dependency view and accelerated structure viewer in Xcode 14.

Core Content

The bottleneck in modern gaming and professional graphics applications isn’t just rendering. Texture streaming loading will be stuck in a large number of small file requests; creating the pipeline state object for the first time may cause dropped frames; high-resolution rendering requires more pixel budget; complex geometry processing often requires running a compute pass first and then writing the intermediate results back to the video memory.

Metal 3 breaks these problems into several new capabilities that can be implemented: resources are loaded into the Metal command model, shader binaries are generated in advance during the build period, MetalFX uses lower resolutions in exchange for high-quality output, and Mesh Shaders puts geometry processing inside the rendering pipeline.

This session is a Metal 3 overview. It does not expand on the complete usage of each API, but explains the specific performance problems solved by each capability and points to subsequent special sessions.

Detailed Content

Metal 3 Device Capability Check

(00:01) Before using the Metal 3 function, first check whether the device supports it.metal3 family。

if device.supportsFamily(.metal3) {

    // My awesome Metal 3 renderer

}

Key Points: -deviceyesMTLDevice

• supportsFamily(.metal3)Query whether the current GPU supports the Metal 3 feature set
• Place Metal 3 exclusive render paths inside conditions
• Need to retain the downgraded rendering path when not supported

Fast resource loading

(01:25) Metal 3 fast resource loading turns many small file loading requests into Metal commands. Each request is a command that can be queued for asynchronous submission and loaded directly into Metal buffers and textures.

The traditional process of texture streaming loading has four steps: deciding to load content based on the feedback of the previous frame, loading tiles from the file, copying from the staging area to the sparse texture, and drawing the current frame. The slower the loading and copying, the longer the application takes to draw with low-quality textures. Fast resource loading reduces intermediate steps and allows the storage hardware queue to maintain sufficient request volume.

Offline compilation

(03:36) Traditional Metal pipelines generate GPU-specific shader binary when created. This operation may occur during the application startup loading phase, or may occur when a new pipeline is created within the frame, resulting in first startup wait or frame rate jitter.

Metal 3 offline compilation moves shader binary generation to project build time. It is faster to create pipeline state objects at runtime, and there are fewer lags when entering the scene for the first time and dynamically creating pipelines.

MetalFX Upscaling

(06:04) MetalFX Upscaling provides platform-optimized super-resolution and anti-aliasing. The app generates pixels at a lower resolution, and MetalFX generates a high-quality, high-resolution image. It supports a combination of temporal and spatial algorithms with the goal of achieving higher frame rates at lower costs.

Mesh Shaders

(07:03) In the traditional graphics pipeline, vertices are independently read, transformed, and output, and then primitives are assembled by fixed-function hardware. When doing GPU-driven culling, LOD selection or procedural geometry, a compute pass is often required to generate a variable number of intermediate geometries and write them into device memory.

Metal mesh shaders replace the traditional vertex stage with a two-stage model: the Object stage first analyzes the entire object to determine whether to expand, contract, or refine; the Mesh stage generates the actual geometry. Both stages run within a render pass to avoid intermediate geometry being written back to video memory.

Ray Tracing, Machine Learning and Tools

(09:34) Metal 3 optimizes acceleration structure build, intersection and shading. After Indirect Command Buffer supports ray tracing, culling on the CPU can be moved to the GPU.

(10:36) Metal 3 also accelerates machine learning. On Mac Studio M1 Ultra, TensorFlow training can be accelerated up to 16 times compared to CPU; after PyTorch uses Metal GPU acceleration, BERT training can be accelerated by 6.5 times, and ResNet50 can be accelerated by 8.5 times.

(13:20) Xcode 14’s Metal Dependency Viewer can view the entire renderer or a single pass and display synchronized edges. Acceleration Structure Viewer can highlight primitives, view primitive data, and visualize motion information at different time points.

Core Takeaways

• Improve texture streaming loading for open world games: Use fast resource loading to directly load tiles into Metal textures to reduce staging copies. When the scene moves quickly, low-quality textures can be replaced faster to improve picture stability.

• Move pipeline compilation out of the first frame: Use offline compilation to generate shader binary during build time. The waiting time for batch creation of pipeline state objects at startup is shortened, and frame drops are reduced when the game enters a new scene for the first time.

• Exchange MetalFX for higher frame rates: Render 3D scenes at a lower resolution on a Retina screen, and then use MetalFX to output high-resolution images. Ideal for racing, action games and real-time preview tools.

• Move geometry clipping to Mesh Shaders: In large-scale scenes, use Object stage to first determine whether the object needs to be expanded, and use Mesh stage to generate visible geometry. LOD selection for procedural terrains, particle swarms, and massive instances.

• Check rendering dependencies with Xcode 14: Use Dependency Viewer to view synchronization edges between passes in complex renderers to avoid hidden waits between fast resource loading, compute and render passes.

Related Sessions

• Load resources faster with Metal 3 — In-depth explanation of Metal 3 fast resource loading
• Target and optimize GPU binaries with Metal 3 — Move shader binary generation to build time
• Boost performance with MetalFX Upscaling — Use MetalFX to improve rendering performance
• Accelerate machine learning with Metal — Accelerate machine learning training with Metal