A bank uses an AI system to approve or reject loans.
A customer gets rejected and asks for the reason.
The bank says: “The model is too complex. We can’t provide an understandable explanation. But it’s very accurate.”
The customer says: “Then I can’t improve my situation. And I can’t check whether it was fair.”
Question:
Is “high accuracy” a sufficient justification for not providing an explanation?
The solution will be unlocked tomorrow.
Solution
No — in many high-impact settings, “accuracy” alone is not enough.
When decisions have serious consequences (loans, housing, jobs, medicine), you also need:
• Explainability (people should understand what happened)
• Contestability (errors must be challengeable and correctable)
• Fairness (systematic disadvantage must be detectable)
• Accountability (someone must be responsible)
Practical approach:
Even complex models can provide human-readable reasons (e.g., feature attribution / explanation methods) or clear process rules (“rejected due to high debt-to-income ratio and short employment history”).
Key point:
A model must not be “too complex” to avoid responsibility.