Capsule Compression includes methods for the efficient representation of capsule outputs in Capsule Networks through dimensionality reduction, selection, or aggregation.
Formally, Capsule Compression can be described as a function \( C: \{ \mathbf{v}_i \}_{i=1}^n \to \{ \mathbf{u}_j \}_{j=1}^m \) with \( m < n \), where \( \mathbf{v}_i \in \mathbb{R}^d \) are the original capsule vectors and \( \mathbf{u}_j \in \mathbb{R}^k \) are the compressed representations.
Techniques include, for example, Principal Component Analysis (PCA) on capsule vectors, weighted sums with adaptive routing weights, or sparse coding to eliminate redundant information.
This compression preserves essential pose and activation information, which is crucial for hierarchical recognition in Capsule Networks.
Definition:
“Capsule Compression refers to methods for reducing the number or dimension of capsules in Capsule Networks to minimize computational effort and memory requirements while preserving the essential features of the data.”
Source:
Wang, X., Liu, M., & Li, J. (2021). Efficient Capsule Compression for Deep Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).