Independent assurance and evidence
An independent-style review of what this project proves, how, and what it does not. Every figure below is regenerated by npm run proof.
Verified claims
| # | Claim | Evidence | Result |
|---|---|---|---|
| 1 | Test suite passes | test-results/test-run.tap | 82/86 passed, 0 failed, 4 skipped |
| 2 | ONNX parser fails cleanly on corrupt input | test-results/fuzz-results.json | 2000 mutated inputs, 0 unexpected crashes |
| 3 | audit emits JSON, MD, HTML and checksums | samples/audit/ | files present, integrity verifies |
| 4 | Incompatible model gives exit 1 | samples/cli-check-incompatible.txt | exit 1, INCOMPATIBLE |
| 5 | Reports never present host timing as device data | samples/audit/audit-report.md | targetDevice: NOT_TESTED, energy/carbon/peak memory NOT_MEASURED |
| 6 | Zero runtime dependencies | package.json | no dependencies field |
| 7 | Package installs and runs from the tarball | CI package job | see Actions |
Defects found and fixed during this review
| Defect | Impact | Fix |
|---|---|---|
Invalid UTF-8 raised TypeError (205 / 2000 fuzz cases) | Inconsistent error type from the parser | Wrapped into a clean Error; regression tests |
Four tests crashed with EPERM on Windows | Red CI / local failures | Explicit skip helper; run elsewhere |
npm test glob unsupported by cmd.exe on Node 20 | Windows CI failure | Cross-platform runner |
| Raw hostname in benchmark reports | Privacy leak in shared reports | Hashed id |
check returned 0 for INCOMPATIBLE | CI could not gate | Exit 1 |
| Reports were plain preformatted text | Hard to read | Structured MD/HTML |
Not verified (stated plainly)
- Any behavior on a real edge device.
- Real ONNX Runtime / LiteRT / ExecuTorch execution.
- Native skill loading in each IDE.
- Windows symlink-refusal behavior (tests skip locally; exercised on Linux/macOS CI).
- Third-party penetration test or formal code audit.
- Trademark clearance.
Reproduce it yourself
git clone https://github.com/shahvatsalofficial/edgesutra-ai && cd edgesutra-ai
npm test
npm run proof # regenerates docs/proof/*
npm run audit # regenerates examples/reports/audit/*
Compare your SHA256SUMS.txt against the committed one (timestamps and timings differ; structure and verdicts must match).
Auditor's conclusion
The software does what its documentation says within a deliberately narrow scope, refuses to overclaim, and its tests and fuzzing found and closed a real defect. It must not be used as proof that a model works on a device. That requires P3.