Architecture

Design pillars

  1. Measurement honesty: every value carries a kind.
  2. Comparability: refuse or downgrade unfair comparisons.
  3. Offline first: no network, no key, no account.
  4. Provenance: hashes, run IDs, raw samples.
  5. Human control: nothing installs, spends or deploys by itself.
  6. Portable skills: plain SKILL.md for any host.
  7. Small trusted core: standard library only.

System context

User / CI / IDE agent
        |
        v
  wattsutra CLI  ---->  core engine  ---->  reports (JSON / MD / HTML / CSV)
        |                  |  ^
        |                  |  +-- fixtures, manifests (user files, read-only)
        |                  v
        |             audit module  ---->  audit.json / audit.md / audit.html
        |
        +--> install  ---->  .claude/ .cursor/ .agents/ .gemini/ .github/ skills (project-local copies)
        +--> server   ---->  127.0.0.1, bearer token, read-only JSON

Dependencies point inward. The core never imports the CLI, server or any vendor SDK.

Components

ComponentResponsibilityTrust
Workload (primes)Fixed allowlisted computationTrusted code, untrusted parameters (bounded)
Recorder (benchmark)Warmup, sampling, environment captureTrusted
Validator (verifyRun, validateRunShape)Integrity and consistencyTrusted; treats files as hostile
Comparator (compare)Parity, quality, gates, intervalTrusted
Estimator (estimate)Explicit-input modelingTrusted; refuses defaults
ReporterEscaped, labelled outputTrusted
Auditor (runAudit)Re-proves invariants liveTrusted
InstallerProject-local skill copyTrusted; refuses unsafe paths
ProbeFile-existence instrument scanTrusted; executes nothing
ServerRead-only localhost APIHardened but not production
SkillsInstructions for AI hostsAdvisory text

Agent model

Twenty roles are instructions, not processes. The Chief Orchestrator (wattsutra-prime) sequences them: scope, comparability (Workload Architect, Fairness Auditor), privacy check, collection (Probe Pilot and instrument roles), quality gate, variance review, bounded modeling (Carbon, Cost), evidence check (Ledger), then report. Evidence and privacy roles may block a claim. The sequence is deterministic and works without any multi-agent runtime.

17 roles are core because tested CLI behaviour backs them; 3 are advisory (GPU scout, FinOps auditor, optimization planner) because no code backs them yet. The distinction is recorded in agents/manifest.json and docs/AGENT_CATALOG.md, generated from one table by scripts/gen-skills.mjs so files cannot drift (npm run check:skills).

Data flow for a fair comparison

manifest.json --bench--> run A (samples, hash)   manifest.json --bench--> run B
        run A, run B --verify--> shape + hash OK
        --compare--> quality equal? env equal? n>=5? interval excludes 1?
        --> gates {latency, energy NOT_RUN, carbon NOT_RUN, cost NOT_RUN}
        assumptions.json --estimate--> modeled energy/carbon/cost (labelled)
        --report/audit--> JSON + Markdown + HTML

Trust boundaries

BoundaryControl
User file into enginesize cap 4 MiB, regular file only, symlink refused, JSON parse errors surfaced
Manifest into workloadallowlist of workload and variant, numeric bounds
Engine into diskexplicit paths, mode 0600, parent dirs created
Network into serverloopback bind, bearer token, GET only, fixed path pattern
Skills into IDE projectproject-local only, no overwrite of edits, no symlinks
Report into browserHTML escaping and CSP meta

Key decisions (ADR summary)

#DecisionReasonCost
1No energy value without a meter or explicit modelCore honesty ruleReports look sparse
2Zero runtime dependenciesSupply-chain and audit simplicityHand-written stats and validators
3Store raw samples in the runAllows recomputation and bootstrapLarger files (max 60 samples per metric)
4Bootstrap interval, not p-valuesFew assumptions, deterministic with seedNot a significance test
5Gate REVIEW_REQUIRED as the default for weak evidenceMissing data must not passMore manual review
6Skills generated from one tableNo drift between skills, agents, manifestEdit the generator, not the files
7Passive probe, no executionNo privilege, no side effectsCannot confirm a tool works
8Fixed workload in 0.2Safe without a sandboxNot representative of real AI workloads

Future architecture (not built)

Instrument adapter interface, sandboxed runner (container, no network, CPU/memory/time caps), billing and trace importers, Sigstore signing, authenticated multi-tenant service, accessible dashboard. Each needs its own threat model update first; see security and legal.