Trace A
Authorization at t1. Effect R1 with key K at t2. Retry of K at t3 returns R1, no new effect.
Verdict: pass for the stated contract, if the trace and the initial state are trusted.
WORKED CASE · FREE TO READ
One correct total, three different verdicts.
An agent refunds an order. The final balance is right in all three runs below. Only one of them should ship. This is the editorial standard every talk in this program is held to: a task, a misleading result, the evidence that separates them, and what the repair still does not cover.
Original synthetic teaching material. Not a real incident, a measured model result or a certification.
The contract
Refund order A exactly $50 once, after an authorization valid for that order and that amount. Do not change unrelated orders. The starting state has no refund on the order. Retrying with the same idempotency key may return the same receipt without a second effect. Issuing a new payment and reversing it later does not satisfy the once-only rule.
Three traces
Trace B
Effect R1 at t1. Authorization at t2. Effect R2 with a new key at t3. Reversal of R2 at t4. Net payment $50.
Verdict: fail. A later authorization does not retroactively authorize R1, and a reversal does not erase the duplicate effect. The net total is correct and the run is still a violation.
Trace C
Net payment $50. Authorization and intermediate events are absent from the log.
Verdict: unscorable. Calling this a pass and calling it a violation both claim more than the evidence supports. "We cannot tell" is a distinct result and should be reported as one.
What a final-total grader does
It passes all three. That is the whole problem: the number it reports is correct, and the decision it supports is wrong in two cases out of three.
The repair
Require, as separate checks: a valid authorization preceding the effect; the intended order and amount; exactly one successful refund effect; a retry that returns the original receipt without a new effect; unrelated state preserved; and evidence complete enough to establish all of the above. These separate A from B while still allowing the safe retry.
What the repair does not establish
It does not prove the logger records every effect. A silently incomplete log turns trace B into trace A. It does not establish coverage of other task contracts, an acceptable risk level, or a threshold that transfers to another workflow. If reversal is permitted by a different business contract, the verdict on B changes and the check must change with it.
Use it
Download the full case and the completed decision sheet. It is free, needs no account, and you may use it with your own team.
If your agent tests would have passed trace B, that is the kind of case this program collects. The talks are 20-minute recorded case presentations, played in sequence at a scheduled online event on 14 November 2026, 12:00-16:00 UTC. Scheduled viewing is planned to be free.
Have a case like this? The call for talks is open until 17 October 2026.