WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Speedrun your game port with agentic coding

Speedrun your game port with agentic coding

Game Porting ToolkitMetal 4AI AgentsMetalFXmacOS 27
Watch original video

Highlight

Game Porting Toolkit 4 introduces agentic skills that give AI coding assistants game-porting expertise, reducing the time needed to port D3D12 games to Metal 4 from months to days while enabling autonomous GPU rendering diagnosis with the gpudebug tool.

Core Content

Porting a D3D12 game to Apple platforms traditionally involves many steps: estimating the effort, getting the project running, converting shaders, bringing up the renderer, remapping input, adding native platform experience refinements, and tuning performance. Each step requires deep Metal knowledge, and the full process can easily take months.

Game Porting Toolkit 4 provides a new set of agentic skills that give your AI coding assistant the expertise needed for porting. These skills fall into two categories: expert skills provide technical guidance, while workflow skills provide a structured method. The porting assistant agent orchestrates the entire porting process and makes sure the right skill is loaded at the right time.

The demo ports Microsoft’s MiniEngine, an open-source D3D12 engine, from Windows to macOS. The complete workflow includes creating a window, bringing up the renderer with Metal 4, adding game controller support, and integrating MetalFX.

The porting process has three phases. The Discover workflow skill analyzes the codebase, captures a reference trace from the evaluation environment, and asks about your preferences. You and the assistant then plan milestone goals, because porting an entire game usually cannot be completed in one session. After each milestone, the agent runs validation checks: whether the app launches correctly, whether Metal validation passes, whether the visuals are correct, whether the code contains antipatterns, whether there are memory issues, and more.

Metal 4 brings explicit memory management and new command structures. Existing models have knowledge gaps in these areas, and the skills are designed to fill them. For example, constants are passed through buffers in Metal 4, and the skill teaches the agent the recommended allocation pattern. All resources must be registered in a residency set before the GPU can access them; an agent without the skill may skip this step and cause rendering errors.

Shader conversion also has important details. A D3D12 root signature must be mapped to Metal argument buffers through the Metal shader converter runtime. The skill teaches the agent to query argument buffer offsets from the runtime instead of calculating them by multiplying an index by a size. Otherwise, rendering can fail silently when the layouts do not match.

The synchronization model is even more different. D3D12 and Metal 4 use different barrier models, and the skill provides a mapping table. The agent uses the MtlProducerStageFromD3D12 and MtlConsumerStageFromD3D12 functions to map states instead of using broad blanket barriers.

The most notable addition is a pair of new macOS 27 command-line tools: gpucapture captures GPU frames, and gpudebug analyzes captures. These tools let the agent debug rendering problems on its own. In the demo, SSAO works correctly but lighting and textures have issues. The agent loads the debugging rendering issues skill, captures a frame with gpucapture, inspects resource bindings, constants, resource contents, and data flow with gpudebug, and eventually locates and fixes the issue.

The final demo also shows upgrading the Godot engine from Metal 3 to Metal 4. The same workflow, the same skills, and the same validation process complete the upgrade within a few days, proving that skills can scale to production-grade projects.

Details

Installation and startup

(03:31)

The skills and assistant are provided as plugins and installed from the Game Porting Toolkit marketplace on GitHub:

/plugin marketplace add apple/game-porting-toolkit
/plugin install game-porting-skills@game-porting-toolkit

After installation, you can ask the porting assistant for guidance directly. The assistant will lead you through a structured workflow.

Resource management

(10:24)

Metal 4 requires all GPU resources to be registered in a residency set before use:

// With skill
residencySet->addAllocation(texture);
residencySet->commit();
// ...
argumentTable->setAddress(texture->gpuAddress(), bindPoint);

// Without skill
argumentTable->setAddress(texture->gpuAddress(), bindPoint);

Key points:

  • residencySet->addAllocation() adds the texture to residency management.
  • residencySet->commit() commits the registration and ensures that the GPU can access the resource.
  • An agent without the skill may skip this step, preventing the GPU from reading the texture correctly.

Argument buffer offset queries

(11:25)

When using the Metal shader converter, the argument buffer layout may differ from expectations and must be queried from the runtime:

// With skill
IRRootSignatureGetResourceLocations(m_MtlCurIRRootSig, locations);
size_t offset = locations[i].topLevelOffset;

// Without skill
size_t offset = paramIndex * descriptorSize;

Key points:

  • IRRootSignatureGetResourceLocations() obtains the real resource locations from the shader converter runtime.
  • locations[i].topLevelOffset is the actual offset.
  • Manually calculating paramIndex * descriptorSize can be wrong when using the shader converter.

Synchronization state mapping

(12:34)

D3D12 states need to be mapped to Metal 4’s producer-consumer model:

// With skill
m_MtlPendingProducerStages |= MtlProducerStageFromD3D12(OldState);
m_MtlPendingConsumerStages |= MtlConsumerStageFromD3D12(NewState);
// ...
m_ComputeEncoder->barrierAfterStages(
    m_MtlPendingProducerStages,
    m_MtlPendingConsumerStages,
    MTL4::VisibilityOptionDevice);

// Without skill
m_ComputeEncoder->barrierAfterStages(
    MTL::StageDispatch,
    MTL::StageAll,
    MTL4::VisibilityOptionDevice);

Key points:

  • MtlProducerStageFromD3D12() maps a D3D12 state to a Metal producer stage.
  • MtlConsumerStageFromD3D12() maps a D3D12 state to a consumer stage.
  • Precise mapping is more efficient and safer than using a broad StageAll barrier.

Shader reflection queries

(14:24)

Query the actual parameter count from shader reflection instead of hard-coding it:

// With skill
IRShaderReflection* refl = IRShaderReflectionCreate();
IRObjectGetReflection(compiledObj, IRShaderStageCompute, refl);
// ...
s_RootSignature.Reset(4, 2); // Reflection reveals: 4 params

// Without skill
s_RootSignature.Reset(5, 2);

Key points:

  • IRShaderReflectionCreate() creates a reflection object.
  • IRObjectGetReflection() obtains reflection from the compiled shader.
  • Initialize the root signature from reflection results instead of hard-coding the parameter count.

GPU debugging tools

(16:02)

New macOS 27 command-line tools allow the agent to debug autonomously:

# Capture a GPU frame
gpucapture

# Analyze the capture
gpudebug

The agent can inspect resource bindings, constants, resource contents, and data flow, compare the result with the reference capture from the evaluation environment, and systematically locate issues.

MetalFX integration

(21:00)

The MetalFX upscaling skill handles complex integration work such as jitter configuration, motion vectors, and history reprojection. The frame interpolation skill sets up a separate presentation thread, precise timing control, and correct presentation ordering.

macOS 27 expands the Metal HUD so it can display the upscaler’s exposure parameters and jitter sequence information, and it provides overlay options for debugging motion vectors and jitter multipliers.

Key Takeaways

  1. Automated porting validation workflow — Use gpudebug tools to automatically compare GPU captures before and after a port, check whether dispatch calls, pipeline state, and resource bindings match, and automate validation at each milestone.

  2. Metal 4 memory management best-practice checker — Write a static analysis tool for existing projects that scans for resources not registered in residency sets and manual argument buffer offset calculations, then marks the code that needs updating.

  3. Dynamic game controller discovery demo — Build a small app inspired by the Game Controller skill that displays the layout and available inputs of connected controllers in real time, demonstrating how to query capabilities instead of hard-coding them.

  4. MetalFX debugging validation panel — Use the Metal HUD’s new overlay options to build a runtime debug panel that can adjust the jitter multiplier and motion vector scale in real time to help debug upscaling integration.

  5. Shader reflection difference analyzer — Compare D3D12 root signatures with the argument buffer layout generated by the Metal shader converter, and automatically detect potential issues such as parameter count mismatches and offset differences.

Comments

GitHub Issues · utterances