Capsule Routing refers to the iterative process in Capsule Networks where the outputs of lower-level capsules are passed on and weighted to higher-level capsules to form a coherent hierarchical representation.
Formally, in dynamic Routing by Agreement, the weighting (routing coefficients) c_{ij} between a lower capsule i and an upper capsule j is calculated by a softmax over so-called routing logits b_{ij}:
c_{ij} = softmax(b_{ij})
The prediction ŷ_{j|i} is formed by multiplying the output u_i of the lower capsule with a weight matrix W_{ij}:
ŷ_{j|i} = W_{ij} u_i
The input s_j of the upper capsule is then the weighted sum of the predictions:
s_j = ∑_i c_{ij} ŷ_{j|i}
The output v_j of the upper capsule is computed by a nonlinear squashing function applied to s_j.
In multiple routing iterations, the logits b_{ij} are adjusted based on the agreement between ŷ_{j|i} and v_j to refine the routing coefficients.
This method allows capsules to only pass signals to parent capsules that agree with their predictions, modeling a part-whole relationship.
Definition:
Capsule Routing is an iterative mechanism in Capsule Networks where the forwarding of signals between capsules is dynamically weighted based on the agreement of their predictions to model hierarchical and part-whole relationships in the data.
Source:
Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic Routing Between Capsules. Advances in Neural Information Processing Systems (NeurIPS).