Neural activation patterns are the characteristic distributions of activation values within the neurons of a neural network that arise in response to specific input data.
Formally, the activation of a neuron \( i \) in layer \( l \) can be described as a function \( a_i^{(l)} = \phi\left(\sum_j w_{ij}^{(l)} a_j^{(l-1)} + b_i^{(l)} ight) \), where \( \phi \) is the activation function, \( w_{ij}^{(l)} \) are the weights, and \( b_i^{(l)} \) is the bias.
The overall activation patterns across all neurons and layers encode the abstracted features of the input and are crucial for the model's decision-making.
The examination and interpretation of these patterns, for example through activation visualization or sensitivity analyses, provide insights into the internal representation and support debugging and explainability of neural networks.
Definition:
“Neural activation patterns are the specific distributions of activation values within a neural network that arise in response to input data and represent the abstract features of that data.”
Source:
Zeiler, M. D., & Fergus, R. (2014). Visualizing and Understanding Convolutional Networks. In European Conference on Computer Vision (ECCV).