08. Testing and evidence
What is tested
The suite uses Node's built-in test runner (npm test). It has two files: tests/core.test.mjs (capsule, redaction, freshness, install, server, CLI basics) and tests/v2.test.mjs (injection quarantine, budgets, decisions, tasks, conflicts, review gate, reports, server host guard, new CLI commands, argument errors, skill drift, self-audit).
| Area | Representative tests |
|---|---|
| Schema and integrity | Capsule declares schema and digest; tampering detected; key order does not change the digest |
| Privacy | Redaction of credentials, multi-line private keys, connection strings; secret files never captured |
| Prompt injection | Instruction-like line quarantined and absent from render; benign text not quarantined |
| Freshness | Edited, removed and added documents reported |
| Diff | Added, removed, changed sources; decision status changes |
| Analysis | Approval only on explicit status; tasks unverified; conflicts flagged; review gate BLOCK |
| Path safety | Traversal refused; symlink source refused (POSIX); install refuses symlinked parents |
| Server | Short token rejected; 401 without token; 405 for POST; 403 for foreign Host |
| Reports | Hostile heading escaped in HTML; javascript: links dropped |
| CLI | Exit codes, clear errors for missing or malformed arguments, install without overwrite, doctor drift |
Two POSIX-only symlink tests are skipped on Windows. They run on Linux and macOS in CI.
Where the proof lives
evidence/README.md is regenerated by npm run evidence and contains:
- the full test run output and a machine-readable summary;
- an exit-code table comparing actual results to the documented behaviour;
- a redaction and injection proof on a fixture with fake secrets;
- sample outputs: capsule, hand-off, reports, review, decisions, tasks;
- a drift demonstration: edit an ADR, see
STALE, rescan, see the diff; - the self-audit in JSON, Markdown and HTML.
How to reproduce on your machine
git clone https://github.com/shahvatsalofficial/contextsutra
cd contextsutra
npm test
npm run audit
npm run evidence
What is not tested
- Live IDE behaviour: whether Claude Code, Cursor, Codex, Gemini CLI or Copilot loads the skills.
- Real-world repositories at scale. Only small synthetic fixtures are used; no performance numbers are claimed.
- Detection quality for secret formats outside the pattern list.
- Local runs on Linux and macOS. Those come from public CI.
No fabricated numbers
Benchmarks, compatibility claims and CI results are never invented. Anything not run is marked untested.