Capsule Network Dynamic Routing refers to an iterative routing algorithm that optimizes the distribution of activations from lower capsules to higher capsules in a neural network to represent hierarchical relationships between features.
Formally, routing coefficients \( c_{ij} \) between lower capsule \( i \) and upper capsule \( j \) are defined by softmax normalization over routing logits \( b_{ij} \):
\[ c_{ij} = rac{\exp(b_{ij})}{\sum_k \exp(b_{ik})} \]
The output of the upper capsule \( s_j \) is given as the weighted sum of the predictions \( \hat{u}_{j|i} = W_{ij} u_i \):
\[ s_j = \sum_i c_{ij} \hat{u}_{j|i} \]
The routing logits \( b_{ij} \) are iteratively updated based on the agreement between \( \hat{u}_{j|i} \) and the output \( v_j = squash(s_j) \), where the squash function performs nonlinear normalization.
This mechanism enables the network to capture complex spatial and part-whole relationships and enhances the ability to correctly recognize objects even under variations.
Definition:
“Dynamic routing in capsule networks is an iterative mechanism for adaptively weighting connections between capsule layers, modeling hierarchical feature relationships and improving the recognition of complex structures.”
Source:
Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic Routing Between Capsules. Advances in Neural Information Processing Systems (NeurIPS).