Test Report

Machine-generated figures from the recorded run. Regenerate with npm run proof.

Environment

FieldValue
ToolLocalSutra AI 0.2.1
Node.jsv24.15.0
Platformwin32/x64
Recorded2026-11-17T17:19:18.302Z

Result

MetricValue
Tests126
Passed126
Failed0
Skipped0
Fuzz cases9500
Unexpected fuzz crashes0

Raw logs: test-run.tap, fuzz-results.json, environment.json, SHA256SUMS.txt.

Test pyramid

LevelExamples
Unitnumeric bounds, statistics, canonical JSON, seal, redaction, URL policy
Boundary / negativemalformed JSON, oversized bodies, remote endpoints, hostile nvidia-smi output, invalid descriptors
Integrationreal loopback servers: healthy, redirecting, hanging, flooding, junk, 503
Filesysteminstaller idempotence, overwrite refusal, symlink and junction refusal, dry-run
Servertoken, method, bind address, tampered report
End-to-endCLI commands via child process, audit file outputs, verify tamper detection
Fuzzseeded hostile model descriptors, GPU text, runtime payloads, seal round-trips

Sample transcripts (real output)

Bugs found and fixed during this release (honest log)

  1. Installer rejected valid Windows project paths that used 8.3 short names (false symlink error). Fixed with segment-wise lstat checks.
  2. Symlink tests failed with EPERM on Windows. Fixed by using junctions.
  3. canonical() handled undefined inconsistently and could break a seal after a JSON round-trip. Fixed and tested.
  4. Markdown tables were not escaping | (caught by the audit's own check E05). Fixed.
  5. A GPU name containing markup was accepted by the first parser draft (caught by check D02). Parser now allow-lists characters.
  6. Report server sent a permissive CORS header and answered POST with 404. Now no CORS header and 405.

What this report does not show

Results on other operating systems (see the CI matrix), live inference on a real Ollama/llama.cpp model (not run), GPU detection on real NVIDIA hardware (parser tested with fixtures; no GPU on the recording host), native IDE skill activation, independent review.