OpenClaw

Eval + Reality Audit

This is the missing reality layer: eval suites, holdouts, replay runs, and audits that can prove whether the machine stayed grounded.

Eval Suites

Purpose-built packs, not vibes.

Scored work the system can point back to.

Judge verdicts across scored cases.

100%

How often the system can do the same safe thing twice.

92%

Every serious behavior should have named tests, not just hope.

1 cases

Outbound Governance Eval Pack

sales_outbound • active

1 cases

Prove the system drafts grounded outreach, respects policy, and does not overstate reality.

Runs recorded: 1

Reality needs a rubric, and regression needs protected difficulty.

1 holdouts

Reality Judge Alpha

reality • active

Score whether the system stayed inside evidence, policy, and operator trust.

High-risk outbound holdout

sales_outbound

Preserve difficult outbound scenarios for future regression checks.

Protected cases: 1

A run should tell you what was challenged and how it ended.

1 runs

lead_follow_up_intent

seeded_baseline • completed

1 scores

Shadow mode outbound recommendation respected policy and requested approval.

Replay is where the system proves it can stay consistent under pressure.

0 failed runs

lead_follow_up_intent

completed

92%

Replay remained inside approval law with no new hallucinated actions.