Overview
An AI review screen can explain why a system made a recommendation and still give the reviewer no practical way to determine whether the recommendation is right. For consequential AI-assisted decisions, CIOs should therefore make verification (not explanation volume) the design requirement.
The near-term decision is straightforward. Before expanding an AI decision-support workflow, require a practical path for the reviewer to inspect decision-critical evidence, challenge the recommendation, and escalate when verification is not feasible.
What Is Happening
AI decision-support designs often collapse four different functions into a single “explanation” layer as shown below:
| Function | Question it answers |
|---|---|
| Recommendation | What does the AI propose? |
| Explanation | Why or how did it reach that recommendation? |
| Evidence | What facts or records matter to judging it? |
| Verification path | How can the reviewer test whether it should be followed? |
These functions can reinforce one another, but they are not substitutes.
Research on human-AI decision making helps explain why. Vasconcelos and colleagues found that …