Pilot measurement protocol. No results are recorded yet.
ID
KPI
Formula
Unit
Data source
Related guideline
Baseline
Follow-up
K1
Accountability names the reviewer and the escalation route.
named reviewer AND route ÷ participants
% of participants
scenario decision
Name the reviewer and the escalation route.
Wk 1
Wk 4
K2
Knowledge spots the misleading output and names the check.
spots misleading output AND names the check ÷ participants
% of participants
written explanation
Check the source before trusting the answer.
Wk 1
Wk 4
K3
Ownership a workflow with named owners and a tested recovery route.
named owners AND tested recovery ÷ 2 workflows
x/2 workflows
workflow note and rehearsal record
Name an owner at each critical handoff, and rehearse recovery.
Wk 1
Wk 3–4
K4
Process the guardrail is applied, and gaps are documented.
guardrail applied AND gaps documented ÷ 2 teams
x/2 teams
work output and adaptation log
Use the override guardrail and record the reason.
Wk 1
Wks 2–4
K5
Library logged gaps are reviewed for the shared library.
adaptations reviewed for the core ÷ 2 teams
x/2 teams
adaptation log
Log where the shared guidance was unclear.
Wk 1
Wks 2–4
Confidence (1–5 self-rating): tracked alongside each decision, not a success measure.
Wk 1Observe the work◆ K1–K5 baseline
Wk 2Build the path
Wk 3Rehearse a failure◆ K3
Wk 4Compare the teams◆ K1–K5 follow-up
Results are reported as counts and percentages, e.g. ‘4 of 6’. Team-level KPIs are reported out of 2. Each follow-up uses a comparable scenario with new case details.
The team, the toolkit, the product
This pilot measures the team. The shared library holds the guidelines. How the AI product behaves with a person is a separate design question.
How this pilot relates to the shared library and to the HAX Toolkit.
The team
The toolkit
The product
Question
Can people review, own and correct AI together?
Do the shared guidelines work, and where must they change?
How should the AI behave with a person?
Where it lives
This pilot, on each team’s real work.
The shared library: guidelines, modules, workbooks and examples.
✗ The gap stays with the team and never reaches the library.
Governance signal Shows whether local adaptations can improve the shared guidelines.
Evidence
adaptation log
Timing
weeks 1–4
Reported as
x of 2
Flex Each team logs its own gap. The core decides what becomes shared.
What leadership sees
Example layout, no data yet
People who know the reviewer and route (K1)
BeforeAfter
—
People who can spot a misleading output (K2)
BeforeAfter
—
Workflows with owners and a tested recovery (K3) x/2
—
Teams using guardrails and logging gaps (K4) x/2
—
Library improvements proposed by teams (K5)
—
The same view works across the company, because the core measures don't change, while each team's scenarios do.
What the results would change
A pattern, and the decision it would lead to. No thresholds.
Pattern
Decision
Both teams struggle with the same item
Update the core library.
One team struggles because of its own workflow
Allow local flexibility.
Nobody can name an owner
Clarify ownership. It isn't a training problem.
Teams adapt a guardrail in the same way
Make the adaptation part of the core.
Confidence rises but decisions don't improve
Revise the practice task.
Confidence is rated 1–5 alongside each decision and reported separately. It is not a success measure. Follow-up scenarios use new case details so people can't repeat a memorised answer.