Capsule Network Explainability refers to the investigation and representation of the internal mechanisms of Capsule Networks (CapsNets), especially the capsule activations and the dynamic routing process, to make the decision-making understandable.
CapsNets consist of capsules that represent vectors or matrices encoding both the probability of a feature and its pose. Information is passed through an iterative routing-by-agreement that dynamically adjusts weighted connections between capsules.
For explainability, methods such as activation visualization, analysis of agreement scores in routing, and decomposition of pose parameters are used to interpret which features and spatial relationships the model uses for classification.
Definition:
“Capsule Network Explainability is the research and application of methods for transparent representation and interpretation of the internal representations and dynamic routing mechanisms in Capsule Networks to make their decision processes understandable.”
Source:
Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic Routing Between Capsules. Advances in Neural Information Processing Systems, 30.