Skip to content

Test Verification & Safety Harness

To ensure zero regressions, eliminate prompt-relic artifacts, and guarantee reliability across production environments, StackPilot enforces a three-tier safety test harness.

1. Multi-Tier Testing Architecture

flowchart TD
    subgraph Tier1 ["Tier 1: Functional Domain Test Suite"]
        A["pytest (212 Tests)\n7 Functional Domains\n63% Code Coverage (5,737 lines)"]
    end

    subgraph Tier2 ["Tier 2: Golden Master Snapshot Harness"]
        B["Byte-for-byte exact equality\n3 Remote Installer Baselines\n(User, Cert, Docker)"]
    end

    subgraph Tier3 ["Tier 3: Headless VM UI Test Suite"]
        C["Node.js VM DOM Sandbox\nToast Alerts, Search Filters,\nSortable Tables, Metric Mini-Bars"]
    end

    Tier1 --> Pass["Production Quality Gate"]
    Tier2 --> Pass
    Tier3 --> Pass

2. Agent Domain Test Suite (src/agent/tests/)

The agent test suite is organized into 7 functional domains:

Domain Files Tests Key Invariants Verified
api/ 11 64 REST endpoints, request dispatching, input validation, and metadata parsing
backup/ 1 5 Safe atomic backup creation, skip-backup flags, and directory preservation
e2e/ 1 1 End-to-end full application deployment and status lifecycle
lifecycle/ 8 77 Image pulling, container restarts, Quadlet compilation, rollback handling
logging/ 2 10 systemd-journald streaming, log ring-buffering, volume logs
monitoring/ 2 12 60-second watchdog timer, local socket liveness, self-heal recovery
system/ 6 43 Rootless runtime detection, firewall UFW rules, port conflicts, storage mounts
Total 31 212 100% Passing Tests (0 Regressions)

3. Golden Master Snapshots (src/hub/tests/snapshots/)

To prevent accidental syntax breaking changes during installer template refactoring: - Three golden master baselines (install-user.snapshot.sh, install-cert.snapshot.sh, install-docker.snapshot.sh) capture the exact byte-level output of the installer script. - tests/agent-source.test.js asserts strict byte-for-byte equality before allowing code changes to pass CI.

4. Headless DOM UI Sandbox (src/hub/tests/ui-components.test.js)

StackPilot's frontend is unit-tested in Node.js without requiring headless browsers (like Chrome or Selenium): - Uses Node's vm module with a lightweight Mock DOM to simulate user interactions. - Verifies toast notification animation exits, status-pill tag filtering, search queries, sortable column comparators, and metric mini-bar thresholds.