07 · Evaluation & Reliability · Topic 02

What are Failure Modes?

Failure modes are the ways an AI system can produce an incorrect, unsafe, or unusable result.

What is it?

A failure mode is a specific way an AI system can fail or create an unacceptable result.

Examples include hallucination, omission, unsafe action, wrong tool use, refusal, bias, timeout, and invalid format. Naming failure modes helps teams test and control them.

Think of it like...

Think of it like inspecting weak points in a machine.

A safety review asks how a machine could fail before an accident happens. Failure-mode analysis asks the same question of an AI workflow.

How it works

1 · MapDescribe the workflowList inputs, decisions, tools, and outputs.

2 · PredictList possible failuresAsk how each step could go wrong.

3 · PrioritizeAssess riskConsider likelihood, impact, and detectability.

4 · TestRun casesUse targeted examples and monitoring.

5 · ControlReduce the riskAdd validation, fallback, review, or limits.

Real-world examples

Business example · Missing evidence

Input: a question with no matching source

System: detects weak evidence and asks for clarification

Output: no fabricated answer

AI product example · Wrong tool action

Input: an ambiguous request to update data

System: validates fields and requires confirmation

Output: a safe action or human review

What it is NOT

Failure Mode ≠ Failure Handling

Failure mode: describes how the system can fail.

Failure handling: defines what the system should do next.

Failure Mode ≠ Random Mistake

Failure mode: is a recognizable pattern of risk.

Random mistake: may not reveal a repeatable cause.

Failure Mode ≠ Bug Only

Failure mode: can come from model, data, process, or integration.

Bug: is one possible implementation defect.

Remember this

Naming failure modes makes AI risks specific enough to test and control.

Independent explainer

Video

visual explainer