Explore numerical computing in Swift

Highlight

Apple releases Swift Numerics and Float16, making Swift numerical algorithms availableRealThe protocol covers Float, Double, Float80 and Float16, and usesComplexTypes directly interface with the complex memory layout of C, C++, and Accelerate.

Core Content

When writing numerical code, it is easiest to start withDoublestart. A probability function, a logarithmic function, and a matrix tool can be run first. Problems will arise when the project becomes larger: some callers wantFloat, some platforms haveFloat80, I want to try the new data pipeline again.Float16. If you duplicate the implementation for each type, the bug fix can easily miss one of them.

This session begins with alogitFunction entry.Darwin.logandDarwin.log1pcan writeDoubleversion, but cannot express “This function is applicable to all floating-point types that truly support logarithmic operations.” Swift generics require a more precise constraint. The entrance given by Swift Numerics isRealProtocol (00:55, 02:32).

RealPut the standard libraryFloatingPointand in Swift NumericsAlgebraicField、RealFunctionsCombined. Developers do not need to disassemble and understand each underlying protocol on a daily basis. They only need to constrain the generic parameters toReal, you can call logarithms, trigonometric functions, power functions and common mathematical functions. The algorithm written in this way will automatically cover the standard floating point types, and will also keep up with new floating point types in the future (03:20, 06:08).

The talk then falls back on the same set of designs for complex numbers and half-precision floating point.ComplexIt is an ordinary Swift struct that saves real and imaginary components, but its memory layout is consistent with C and C++ complex types. You can pass the Swift array pointer to the BLAS function of Accelerate.Float16Then enter the Swift core language and standard library, which are available on the ARM platform and comply withReal. This makes a batch already usedRealAlgorithms have been written to handle 16-bit floating point without source code changes (07:03, 11:28).

Detailed Content

1. UseRealWrite a logit

(01:05) The original implementation only acceptedDouble. It’s suitable for demonstrating formulas, but it hard-codes genres.

import Darwin

/// The log-odds function
///
/// https://en.wikipedia.org/wiki/Logit
///
/// - Parameter p:
///   A probability in the range 0...1.
///
/// - Returns:
///   The log of the odds, 'log(p/(1-p))'.
func logit(_ p: Double) -> Double {
    log(p) - log1p(-p)
}

Key points:

• import DarwinProvided in the C math librarylogandlog1p。
• func logit(_ p: Double) -> DoubleFix the input and output toDouble。
• log(p) - log1p(-p)It is a numerically stable way of writing log-odds to avoid direct calculation.log(p / (1 - p))。
• This code cannot be used directlyFloat、Float80Or new floating point types added in the future.

(02:33) After introducing Swift Numerics,logitCan be changed to a generic function. The constraint changes from a concrete type toNumberType: Real。

import Numerics

/// The log-odds function
///
/// https://en.wikipedia.org/wiki/Logit
///
/// - Parameter p:
///   A probability in the range 0...1.
///
/// - Returns:
///   The log of the odds, 'log(p/(1-p))'.
func logit<NumberType: Real>(_ p: NumberType) -> NumberType {
    .log(p) - .log(onePlus: -p)
}

Key points:

• import NumericsIntroducing the Swift Numerics package.
• NumberType: RealIndicates that this type is a complete floating-point real number type that supports standard arithmetic and mathematical functions.
• .log(p)Generic math functions exposed using Numerics are no longer bound to Darwin’sDoublefunction.
• .log(onePlus: -p)corresponding to the originallog1p(-p), retaining a more stable calculation method close to 0.
• The return type is stillNumberType, the caller passes inFloatJust getFloat, pass inDoubleJust getDouble。

2. RealTie up floating point capabilities into a constraint

（03:20）RealThe value lies in clear boundaries. Its meaning is very specific: a type that supports complete floating point mathematics. The talk mentioned that it combines multiple protocols from the standard library and Numerics.

func logit<NumberType: Real>(_ p: NumberType) -> NumberType {
    .log(p) - .log(onePlus: -p)
}

Key points:

• FloatingPointProvides floating point comparison, exponent and significant digit splitting, infinity,piand other basic abilities.
• AlgebraicFieldexistSignedNumericDivision is added above to express the complete number system of the four arithmetic operations.
• ElementaryFunctionsCovers trigonometric functions, logarithms, exponents, roots and powers.
• RealFunctionsExpanded to gamma, error functions, and more trigonometric function variations.
• Everyday algorithms are usually written directlyReal, it is only necessary to deeply split these protocols when implementing new numeric types.

3. ComplexExpressing complex numbers with Swift struct

(07:10) Swift Numerics provides a complete set of plural types. The minimum usage is straightforward: import Numerics and construct a real/imaginary combination.

import Numerics

let z = Complex(1.0, 2.0) // z = 1 + 2 i

Key points:

• Complex(1.0, 2.0)Creates a complex number with real part 1 and imaginary part 2.
• The default for both literals isDouble, so herezwill be inferred asComplex<Double>。
• ComplexIt is a generic type itself, which requires that the underlying numerical type conforms toReal。

(07:38) The speech showsComplexThe basic shape: two fields, an initializer.

public struct Complex<NumberType> where NumberType: Real {
    /// The real component
    public var real: NumberType

    /// The imaginary component
    public var imaginary: NumberType

    /// Construct a complex number with specified real and imaginary parts
    public init(_ real: NumberType, _ imaginary: NumberType) {
        self.real = real
        self.imaginary = imaginary
    }
}

Key points:

• where NumberType: RealLet both the real and imaginary parts have full floating-point math capabilities.
• realSave the real part,imaginarySave the imaginary part.
• The initializer does not hide extra state, a complex number is a combination of two floating point values ​​of the same type.
• This layout also lays the foundation for subsequent C and C++ interoperability.

4. Complex number operations and polar coordinate dependenceRealmathematical function

（08:04）Complexcan conform toSignedNumeric, because it defines addition, subtraction, and multiplication.

extension Complex: SignedNumeric {
    /// The sum of 'z' and 'w'
    public static func +(z: Complex, w: Complex) -> Complex {
        return Complex(z.real + w.real, z.imaginary + w.imaginary)
    }

    /// The difference of 'z' and 'w'
    public static func -(z: Complex, w: Complex) -> Complex {
        return Complex(z.real - w.real, z.imaginary - w.imaginary)
    }

    /// The product of 'z' and 'w'
    public static func *(z: Complex, w: Complex) -> Complex {
        return Complex(z.real * w.real - z.imaginary * w.imaginary,
                       z.real * w.imaginary + z.imaginary * w.real)
    }
}

Key points:

• +Add the real and imaginary parts of two complex numbers respectively.
• -Subtract the real and imaginary parts of two complex numbers respectively.
• *Using the standard complex multiplication formula, the real and imaginary parts are cross-multiplied.
• SignedNumericletComplexAccess to more generic numerical algorithms.

(08:19) The polar coordinate interface showsRealThe collection of functions brought:hypot、atan2、cos、sincan be used in genericsNumberTypeused on.

extension Complex {
    /// The Euclidean norm (a.k.a. 2-norm) of the number.
    public var length: NumberType {
        return .hypot(real, imaginary)
    }

    /// The phase (angle, or "argument").
    ///
    /// Returns the angle (measured above the real axis) in radians.
    public var phase: NumberType {
        return .atan2(y: imaginary, x: real)
    }

    /// A complex value with specified polar coordinates.
    public init(length: NumberType, phase: NumberType) {
        self = Complex(.cos(phase), .sin(phase)).multiplied(by: length)
    }
}

Key points:

• lengthuse.hypot(real, imaginary)Calculate the Euclidean norm.
• phaseuse.atan2(y:x:)Calculates complex phase angles.
• init(length:phase:)Use firstcosandsinGet the unit direction and scale it by length.
• These functions are derived fromRealconstraints, the code does not need to care about the underlyingFloatstillDouble。

5. Swift plurals can be passed directly to Accelerate

（09:16）Complexis an ordinary struct containing only two floating point values. The talk clearly states that its memory layout matches C and C++ complex types, so Swift buffers can be passed to the corresponding C or C++ library. The example uses Accelerate’s BLAS function to calculate 2-norm.

import Numerics
import Accelerate

/// Array of 100 random Complex<Double> numbers
let z = (0 ..< 100).map {
    Complex(length: 1.0, phase: Double.random(in: -.pi ... .pi))
}

/// Compute the Euclidean norm of z
let norm = cblas_dznrm2(z.count, &z, 1)

Key points:

• import AccelerateIntroducing Apple’s high-performance numerical libraries.
• zis 100Complex<Double>Array consisting of each value created by length and random phase.
• &zPass Swift arrays as pointers to the C API.
• cblas_dznrm2Receives an array of complex numbers and returns the Euclidean 2-norm.
• The speech reminded that when porting C or C++ code, you should pay attention to the difference between infinity and NaN conventions; the convention adopted by Swift is simpler and also brings better performance.

6. Float16 trades 16 bits for memory and throughput

（11:28）Float16It is Swift’s new IEEE 754 floating point format. It conforms to the standard library core protocol and also conforms toReal. This is crucial: it has been writtenNumberType: Realalgorithm does not need to beFloat16Make a separate copy.

func logit<NumberType: Real>(_ p: NumberType) -> NumberType {
    .log(p) - .log(onePlus: -p)
}

Key points:

• Float16It occupies 16 bits, which is 2 bytes.
• FloatOccupies 4 bytes,DoubleOccupies 8 bytes.
• Smaller elements allow more values ​​to be accommodated in the same SIMD register or the same page of memory.
• At the cost of smaller precision and range, the maximum representable value is slightly over 65,000, migrationFloatorDoubleCode should be checked for overflow risks.
• Apple GPU has long-term support for half-precision, and Apple CPU has direct support starting from A11 Bionic; older CPUs will useFloatSimulation, same results but slower.

(14:07) Lecture gives performance numbers using BNNS convolution benchmark: single precisionFloatAbout 49 giga-flops,Float16About 119 giga-flops. This example is suitable for workloads such as machine learning, image processing, and signal processing where half-precision errors are acceptable.

Key points:

• First confirm whether the algorithm can withstandFloat16accuracy and range.
• Then write the core generic constraints asReal,letFloat16Shares the same source code with other floating point types.
• Finally, use the actual device benchmark to verify the benefits, because half precision will be simulated on old CPUs.

Core Takeaways

1. Put the items in the projectDoubleThe tool function is changed to a generic numerical function

What to do: Find tool functions such as probability, statistics, curves, interpolation, etc., and accept onlyDoubleThe interface is changed toNumberType: Real。

Why is it worth doing: sessionlogitExample showing copyingDouble、Float、Float80version maintenance costs;RealThese implementations can be combined.

How ​​to start: First introduce Swift Numerics and change the function signature tofunc f<NumberType: Real>(_ x: NumberType) -> NumberType, and then replace the Darwin mathematical functions with the generic form of Numerics, for example.log、.log(onePlus:)、.hypot。

2. Add complex pipelines to audio or signal processing apps

What to do: UseComplex<Double>orComplex<Float>Represent frequency domain data, and then pass the buffer to Accelerate to calculate the norm or linear algebra result.

Why It’s Worth Doing: Talk Notes SwiftComplexThe memory layout matches C and C++ complex types and can be directly connected to Accelerate’s BLAS API.

How ​​to start: Use firstComplex(length:phase:)Construct test data and then usecblas_dznrm2This type of function verifies the pointer transfer and result; after verification, the real FFT or filter result is connected to the same structure.

3. Experiment with Float16 for images or machine learning paths

What to do: Change convolution, embedding, pixel batching or feature caching that can tolerate half-precision errors to supportFloat16。

Why is it worth doing: In the BNNS benchmark given by session,Float16convolution increases from ~49 giga-flops to ~119 giga-flops; half precision also reduces memory usage.

How ​​to start: First take a commission from the core calculationRealGeneric function, then run separately using the same set of inputsFloatandFloat16;Record errors, overflow times, and elapsed time on the target device.

4. Split protocol implementation for custom numeric types

What to do: If the item has a fixed point, angle, probability, or decimal type, pressAdditiveArithmetic、SignedNumeric、AlgebraicField、RealFunctionshierarchical design capability boundaries.

Why it’s worth doing: Speech explainsRealIt is composed of multiple sets of protocols. After clearing the boundaries, callers can only rely on the capabilities they really need.

How ​​to start: First implement addition, subtraction and multiplication, and then determine whether division, trigonometric functions, exponents and logarithms are really supported; only when it has complete floating point semantics, then consider complianceReal。

Related Sessions

• What’s new in Swift — Check out Swift 5.3’s overall introduction to Float16, Package ecosystem, runtime, and language features.
• Embrace Swift type inference — Understand how Swift infers from generic constraints and call-site cluesNumberTypeSuch type parameters.
• Safely manage pointers in Swift — Supplement the memory safety rules that need to be mastered when passing Swift array pointers to the C API.
• Distribute binary frameworks as Swift packages — Understand the role of Swift Package in dependency distribution, and use it with packages such as Swift Numerics.