Explainable Reinforcement Learning (XRL) refers to a research field that aims to make the decision-making processes of Reinforcement Learning (RL) agents interpretable and comprehensible.
Formally, an RL agent is defined by a Markov Decision Process (MDP) \( M = (S, A, P, R, \gamma) \) with state space \( S \), action space \( A \), transition probabilities \( P \), reward function \( R \), and discount factor \( \gamma \). The goal is to learn a policy \( \pi: S \rightarrow A \) that maximizes the expected cumulative return.
XRL methods attempt to generate explanations \( E \) that provide insights into the structure, values, or decisions of the policy \( \pi \). Examples include feature attributions, saliency-based visualizations, symbolic explanations, or model compression into more understandable forms.
Such explanations are essential for the acceptance and safety of RL systems in areas like autonomous driving, robotics, or healthcare.
Definition:
“Explainable Reinforcement Learning is a subfield of machine learning that aims to make the decisions of RL agents understandable through transparent and comprehensible explanations, in order to increase trust and control.”
Source:
Madumal, P., Miller, T., Sonenberg, L., & Vetere, F. (2020). Explainable Reinforcement Learning Through a Causal Lens. Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI).