WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
XCTSkip your tests

XCTSkip your tests

WWDC 2020XCTestDeveloper Tools
Watch original video

Highlight

XCTestXCTSkipAPI (introduced in Xcode 11.4) lets tests return explicit skip results when runtime conditions are not met, and Xcode and CI reports document skip location, reason, and device dimension results.

Core Content

Integration testing often relies on the runtime environment. One test might only work on iPads because the feature is only enabled on iPad; another test might call an API that’s only available in iOS 13.4; and yet another test might rely on a server that is occasionally maintained. These conditions are difficult to determine at compile time and can only be known when the test is run.

Dealing with this situation has been awkward in the past. directreturnThe test will be displayed as passed, and the report will imply that the code has been verified; forcing failure will treat environmental limitations as a product problem, wasting troubleshooting time.XCTSkipAdded a third result to XCTest: pass, fail, skip.

An example of a session is a small app called Play Garden. It adds a new pointer interaction test, but pointer interaction is only available after iOS 13.4, and the App only enables this interaction on iPad. The test method starts by checking the system version and device type; when the conditions are not met, the test throws skip instead of pretending to pass or fail.

This result will appear in multiple entries in Xcode. Skip comments will be displayed next to the source code, Test Navigator can filter skipped tests, and the test report will list the file, line number, and reason. When the same test is run on three devices in CI, the report can simultaneously show that one iPad passed and the other devices skipped, which is closer to real test coverage than a vague green result.

Detailed Content

1. Turn runtime preconditions into test results

(00:14) The problem is defined at the beginning of the session: some test dependencies cannot be easily mocked, such as device types, system versions, and external service status.XCTSkipLet the test author declare “not executed this time” at runtime and write this status into the test results.

(01:20) This API was introduced in Xcode 11.4. Tests can now pass, fail, or be marked as explicit skip. Xcode distinguishes these tests with dedicated skip icons, allowing reports to illustrate what the test suite actually verified.

func testExample() throws {

    /// Example usage: skip test if device is not an iPad
    try XCTSkipUnless(UIDevice.current.userInterfaceIdiom == .pad,
              "Pointer interaction tests are for iPad only")

    // test...
}

Key points:

  • testExample()marked asthrows, because skip will interrupt the current test by throwing results that are recognized by XCTest. -XCTSkipUnlessThe expression isfalsetest is skipped. -UIDevice.current.userInterfaceIdiom == .padCorresponds to the iPad-only pointer interaction scene in transcript.
  • The reason string goes into Xcode and CI reports and team members can see the skip reason.

2. UseguardHandle system version requirements

(02:19) Play Garden’s pointer interaction test also has a system version condition: pointer interaction was introduced in iOS 13.4. There is no point running this test on an old system because the API under test does not exist.

(05:52) session shows throwing directlyXCTSkipHow to write struct. it is suitable for andguardIn combination, use a precondition to block subsequent test logic.

func testExample() throws {

    /// Example usage: skip test if OS version is older than iOS 13.4
    guard #available(iOS 13.4, *) else {
        throw XCTSkip("Pointer interaction tests can only run on iOS 13.4+")
    }

    // test...
}

Key points:

  • #available(iOS 13.4, *)Write the system version requirements in the test entry. -throw XCTSkip(...)The skip result is returned directly, and the test report will not falsely report the old system environment as a failure.
  • reason specifies the skip condition and minimum system version.
  • Subsequent test code will only run in an environment that meets the preconditions, and assertion failures are more likely to represent real product problems.

3. UnderstandXCTSkipIfandXCTSkipUnless

(05:25) There are two throwing functions at the API level:XCTSkipIfandXCTSkipUnless. Their parameters are the same, the difference lies in the trigger conditions.

try XCTSkipUnless(UIDevice.current.userInterfaceIdiom == .pad,
                  "Pointer interaction tests are for iPad only")

Key points:

  • XCTSkipIfThe expression istruetime skip. -XCTSkipUnlessThe expression isfalsetime skip.
  • Both are suitable to be written at the beginning of the test method, first stating the running conditions, and then entering the actual assertion.
  • Which function to choose depends on whether the conditional expression sounds natural.

4. View skip in Xcode and CI

(03:22) When the test is run on an old device, Xcode replaces the green pass icon with a gray skip icon and marks the skip location and reason in the source code.

(03:37) Test Navigator will display the skipped status, and you can use the bottom button to filter to only see skipped tests. The test report will display the file, line number and reason after expansion. The new assistant button in Xcode 12 can open the secondary editor and display reports and source code side by side.

(04:34) CI reporting preserves device dimensions. In the example, the same test is run on three devices. One iPad passes and the other devices skip it. After expanding the results of each device, you can see the corresponding skip position and reason.

try XCTSkipUnless(UIDevice.current.userInterfaceIdiom == .pad,
                  "Pointer interaction tests are for iPad only")

Key points:

  • The same test can get different results on different destinations.
  • skip reason is part of the CI troubleshooting material, and there is no need to go back to the local machine to guess why it did not run.
  • Pass, fail, and skip statistics are separated, so that the test coverage is closer to the actual execution situation.
  • This kind of information is suitable for putting into the result bundle analysis process of pull request or nightly test.

Core Takeaways

  • **What to do: Skip UI testing by device capabilities. ** Why it’s worth doing: The iPad pointer interaction example of session shows that the device type is a runtime condition. How to start: Use at the beginning of the test methodXCTSkipUnless(UIDevice.current.userInterfaceIdiom == .pad, "...")Protection is only suitable for iPad interactive testing.
  • **What to do: Write the minimum system version into the test entry. ** Why it’s worth doing: When the old system lacks the API to be tested, the failure results will mislead troubleshooting. How to start: Useguard #available(...) else { throw XCTSkip(...) }Wrap tests that depend on the new API.
  • **What to do: Track skipped tests in CI reports. ** Why it is worth doing: The same test may have pass and skip on different devices, and the session shows the device dimension results. How to start: Keep the result bundle and press skipped filter to see the skipped number, position and reason.
  • **What to do: Create precondition checks for integration tests. ** Why it’s worth doing: External server maintenance, lack of hardware capabilities, or abnormal account status are all runtime conditions. How to start: Put the check in the first paragraph of the test, and throw it when the conditions are not met.XCTSkip, and then execute the assertion after the condition is met.
  • **What to do: Write skip reason as troubleshootable text. ** Why it’s worth doing: Xcode source code comments, Test Navigator, test report and CI expansion items will all display reason. How to get started: Use a string containing conditions and action clues, such as minimum system version, target device or service status.

Comments

GitHub Issues · utterances