Capsule Network Sparsity refers to the property that only a subset of the capsules within a layer in capsule networks is activated to produce efficient and focused feature representations.
Formally, sparsity can be expressed through a regularizing loss function \( \mathcal{L}_{sparse} = \lambda \sum_j \|v_j\|_1 \), where \( v_j \) is the activation of capsule \( j \) and \( \lambda \) is a regularization parameter.
This sparsity promotes a distribution where few capsules show high activations while others remain near zero, increasing discriminative capability.
Furthermore, sparsity can be implemented through CapsNet architectures with competing routing mechanisms or through sparse routing via masking.
The sparse approach acts as a regularizer and helps prevent overfitting, improves the interpretability of capsule representations, and reduces computational cost.
Definition:
“Capsule Network Sparsity is the property of capsule networks where only a limited number of capsules within a layer are activated to ensure efficient, robust, and interpretable feature representations.”
Source:
Wang, X., Yu, F., Dou, Z., & Tao, D. (2019). Sparse Capsule Networks. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).