IncidentSutra AI — Automated Audit

Every check runs real code against this repository and records what it observed.

verdict: PASS_WITH_NOTESincidentsutra 0.2.1Node v24.15.0win32/x64

Summary

20checks
19pass
0fail
1skipped

Results

IDAreaCheckResultObserved
A01agentsAgent manifest, skills and Claude role files agree (20 roles)PASS20 manifest entries, 20 SKILL.md files, 20 role files; frontmatter and 7 required sections present in each
agents=20
A02agentsEvery skill forbids autonomous production action and defers to humansPASSall 20 skills contain the no-autonomy, human-owner and untrusted-input rules
A03contractsSchemas parse; generated report and every event validatePASSreport + 9 events + agent contract validate
A04determinismIdentical input produces an identical report ID and bytesPASSreportId inc_08ab6bbd890e21fe62bd
A05acceptanceSpec acceptance: time zones sort, correlation stays unconfirmed, absent metrics stay unknown, claims cite sourcesPASS{"sorted":true,"mixedOffsetsPresent":true,"unconfirmed":true,"claimsCiteSources":true,"absentMetricsUnknown":true}
A06privacyRedaction corpus: 14 synthetic secret classes never reach any outputPASS14/14 masked in JSON, Markdown, HTML, postmortem, register and JSONL; 0 redactions applied
known gaps: IPv6 addresses; personal names and free-text identifiers; street addresses; secrets without a recognizable prefix or key name; non-US phone and national-ID formats; secrets split across fields
A07securityHostile markup is escaped in HTML and cannot break Markdown tablesPASSrawTags=false, csp=true, brokenMarkdownRows=0
A08securityPrompt-injection corpus (6 payloads): no state change, nothing executedPASSdecision=HUMAN_REVIEW_REQUIRED; executedFlags=false; verifies=true
A09securityStatic scan: no process spawn, outbound network, eval or dynamic code in product codePASS10 source files scanned; node:http confined to packages/server
A10integrityTampering with a report is detected by verificationPASS6/6 mutations detected; untouched report verifies
A11serverLocal API: loopback only, bearer required, GET only, host-checked, allowlisted pathsPASS{"addr":"127.0.0.1","noAuth":401,"wrongAuth":401,"ok":200,"notFound":404,"post":405,"hostBad":403,"weakTokenRejected":true}
A12ingestInput limits and timestamp strictnessPASSoversizeRejected=true; invalidTimestampsRejected=6/6; leapDayAccepted=true; maxEvents=5000
A13ingestSymlinks are never followedSKIPPEDhost forbids symlink creation (EPERM); covered on Linux/macOS CI
A14algorithmSynthetic benchmark: 26 case familiesPASS26/26 cases, 36/36 expectations. self-authored synthetic labels; not independently validated; measures internal consistency, not real-world accuracy
A15legalLicence, notices and policy files present and consistentPASSApache-2.0 text verified; 0 runtime dependencies; all policy files present
A16legalDocumentation does not overclaim (security, accuracy, compliance, readiness)PASS105 documents scanned; no unqualified overclaims
A17secretsNo credentials committed to the repositoryPASSpattern scan (GitHub, AWS, Slack, npm tokens, private keys) found nothing outside intentional synthetic fixtures
A18supply-chainZero runtime and dev dependencies; engines and bin declaredPASSdependencies=0; engines.node=>=20; bin=packages/cli/index.mjs
A19safetyRead-only policy file denies network, exec, mutation and auto-mitigationPASS{"mode":"read-only","network":false,"exec":false,"remoteWrite":false,"autoMitigation":false}
A20safetyRedaction helper handles nested secrets and keeps benign dataPASS{"authorization":"[REDACTED]","nested":{"apiKey":"[REDACTED]","list":["ok","token=[REDACTED]"]},"latency_ms":220,"note":"v2.7.0 deployed"}

Known redaction gaps

Limitations of this audit