Evidence coverage
How much of a material claim is supported by accessible, immutable evidence?
Benchmark / Methodology first
Measure the review, not the confidence of the prose.
Hyperoru evaluates whether an AI-assisted review is supported, useful, isolated, repeatable, and honestly costed.
Evaluation contract
Six dimensions before one score.
A model cannot compensate for missing evidence, invalid citations, tenant leakage, or an unreplayable workflow. Each dimension has its own hard failures.
Citation validity
Does each factual statement point to the exact evidence that supports it?
Decision usefulness
Can a reviewer understand consequence, uncertainty, and the safest next action?
Replayability
Can the same source, policy, context, agent, and model inputs reproduce the review?
Tenant isolation
Can any retrieval, cache, stream, or export cross a workspace boundary?
Operational cost
Do model, scanner, remediation, and assistant costs reconcile to measured invocations?
Protocol
Publish the test before publishing the result.
The public benchmark will include source fixtures, expected evidence, adversarial instructions, model and prompt versions, timeouts, retries, and raw structured outputs.
Current status
Methodology published. Internal evaluation is active. Comparative scores stay unpublished until the fixtures, model versions, and failure conditions can ship with them.
Results pending reproducibility gateRelease gate
Every factual claim cites accessible evidence.
Cross-tenant canaries never appear in retrieval, responses, caches, or streams.
The same inputs replay without hidden state.
Timeouts, partial results, and coverage gaps remain visible.
Usage and cost reconcile to provider and internal ledgers.
Bring a failure case, not a polished demo.
The best evaluation cases contain missing evidence, conflicting sources, prompt injection, ambiguous ownership, or a decision that is expensive to get wrong.