2026 Dedicated Remote Mac CI: Swift Testing (Macros) vs XCTest Mix — Compile Time, Parallelism, Simulator Quotas & SSH Headless Flaky Decision Matrix + FAQ
On a dedicated remote Mac CI worker, Swift Testing (heavy compile-time macro expansion) and legacy XCTest often coexist for quarters. The goal is not a manifesto to pick one framework—it is to partition compilation, cap parallelism, budget Simulator boots, and enforce repro discipline over SSH. Below is a paste-ready matrix plus FAQ. For lock-file and reproducibility policy, pair this with
SwiftPM Package.resolved on dedicated remote Mac CI;
for pool economics and queues, see
GitHub Actions self-hosted runner billing vs dedicated Mac pools.
1. When the mix hurts first: compile time and macro vs XCTest partitions
Swift Testing leans on macros (for example #expect, @Suite—exact spellings depend on your toolchain). Macro expansion shifts work into the Swift front end and widens incremental rebuild surfaces. Co-locating XCTest and Swift Testing in one test target can mean a tiny edit retriggers both bridging headers and generated Swift Testing scaffolding.
- Default recommendation: give Swift Testing its own
*.xctest/ SwiftPM test target, physically separate from legacy XCTest targets; drive PR vs nightly coverage with a Test Plan on the main scheme. - CI gating: skipping the Swift Testing target on “UI-only” or “ObjC bridge-only” PRs needs explicit path rules or
CODEOWNERSguardrails so you never silently skip real risk. - Pin the toolchain: log
xcodebuild -versionand Swift language mode into the runbook—macro behavior moves with compiler upgrades faster than XCTest-only suites.
2. Compile-time triage matrix
| Symptom | First action | Cost |
|---|---|---|
| Clean builds spend too long in the test phase | Split Swift Testing targets; enable module stability and normalize DerivedData cache keys |
More schemes / Test Plans to maintain |
| Incremental builds still hot-spot macro files | Shrink “giant shared fixtures” imported transitively from macro-heavy tests; audit @testable import fan-out |
Requires code decoupling |
| Nightly needs full coverage; PRs need sub-minute feedback | PR runs a Swift Testing smoke subset; merge queue or main runs full XCTest / combined suites | Layered gates must catch “subset green, full red” |
3. Parallelism: the mix is not “double the threads”
xcodebuild test parallelism multiplies Simulator processes, disk IO, and compiler subprocess fan-out. Running Swift Testing and XCTest in one invocation often contends harder for memory than two sequential single-framework runs because link products and runtime metadata share one user session.
- Baseline: start with
-parallel-testing-enabled NOor a capped-maximum-parallel-testing-workers, record flake rate, then ramp. - Split jobs: run Swift Testing targets in one CI job and XCTest in another (still serialized on one exclusive Mac) for cleaner
xcresultattribution. - Isolate UI tests: never share peak parallelism between UI tests and unit / Swift Testing suites—Simulator boot storms mask real compile regressions.
4. Simulator quotas: exclusive hardware still has ceilings
Exclusive tenancy removes cross-tenant contention; it does not remove CoreSimulatorService limits or memory pressure. Practical concurrent boot caps correlate with RAM, free SSD headroom, and whether you also target watchOS / tvOS.
| Machine RAM tier | Starting concurrent Simulator cap | Notes |
|---|---|---|
| 16 GB | 1–2 heavy UI / mixed suites; pure logic may go slightly higher only after measurement | Stay conservative when Swift Testing + XCTest overlap |
| 24–32 GB | 2–4 depending on destination, screenshots, and screen recording | Watch memory_pressure and SSD latency tails |
| 64 GB+ | Higher ceilings possible, still bounded by Xcode / OS bugs and IO jitter | More RAM ≠ unbounded parallelism |
Document per job: destroy vs reuse booted devices, whether you call xcrun simctl shutdown all, and any dependency on logged-in GUI sessions for keychain access (next section).
5. SSH headless flaky: trim the environment before blaming tests
Driving xcodebuild over SSH often flakes on GUI session shape, keychain unlock prompts, first-boot Simulator races, and GPU paths without a display. These are not logically caused by Swift Testing vs XCTest, but higher parallelism makes them visible.
- Session type: distinguish
launchdagents, non-interactivesshshells, and console-logged-in users—code signing and helper permissions differ. - Keychain: interactive unlock dependencies fail intermittently in CI; use a dedicated CI user, pre-provisioned profiles, and non-interactive secret injection with rotation notes in the runbook.
- Repro bar: replay the exact SSH command for “CI-only red” cases, fix seeds /
SIMCTL_CHILDwhere applicable, and require N consecutive passes before rewriting product or test code.
6. Executive decision matrix
| Constraint | Bias your split toward |
|---|---|
| PR latency sensitive; macros dominate compile | Dedicated Swift Testing targets + PR subsets; full XCTest / combined suites nightly or on merge |
| Stability first; sporadic timeouts | Lower parallel workers; split jobs; serialize Simulator boot |
| SSH headless without GUI login | Fix session / keychain / boot races before scaling parallelism |
| Single source of truth for reports | One xcresult path per job; avoid multi-stage overwrites |
7. FAQ
Q1: Can we migrate everything to Swift Testing and delete XCTest?
As a direction, yes—but CI splits should be phased: coexist, migrate module by module, and keep Test Plan gates explicit. Big-bang switches combine compile-graph churn with flaky attribution chaos.
Q2: Macros are slow—can we fix it with more -j or more runners?
Horizontal scale hides single-target macro fan-out. If the Swift front end is the bottleneck, more parallel jobs often amplifies IO and Simulator contention. Prefer target splits and cache keys first.
Q3: We are on an exclusive Mac but still flaky—are the tests wrong?
Not necessarily. Bisect with the SSH / keychain / Simulator checklist: green under a local GUI user but red only over SSH usually means environment-class flake.
Why Mac mini is a good place to freeze this split
Swift Testing plus XCTest needs a predictable compile graph, a stable user session, and enough unified memory bandwidth for mixed xcodebuild and Simulator workloads. Apple Silicon Mac mini nodes stay efficient under long builds and multi-simulator pressure, which suits exclusive self-hosted runners. macOS gives mature paths for SSH automation, scripting, and keychain workflows compared with opaque hosted Mac pools. On security, Gatekeeper, SIP, and FileVault help you keep CI accounts minimally exposed, and on total cost of ownership the small chassis, quiet operation, and roughly 4 W-class idle power matter when pipelines run around the clock.
If you want one machine to own your parallelism baselines and Test Plans for 2026, Mac mini M4 remains a strong value entry point—start from the home page to compare plans and bake the matrix above into your runbook.