Neural Turing Machines (NTM) are differentiable models consisting of a controller unit, usually a recurrent neural network, and an addressable external memory.
The controller learns through backpropagation to control read and write operations on the memory, enabling the modeling of algorithmic processes.
Formally, reading from the memory can be represented as a weighted sum of the memory cells:
\[ r_t = \sum_i w_t(i) M_t(i) \]
where \( w_t(i) \) are the differentiable read weights and \( M_t(i) \) are the memory contents.
Write operations modify the memory through weighted additions and erasures, also controlled by \( w_t(i) \).
This architecture extends classical neural networks with a dynamic memory component and enables learning complex, iterative algorithms.
Definition:
“Neural Turing Machines are differentiable models that combine a neural network with an addressable external memory to solve algorithmic tasks through learnable read and write operations.”
Source:
Graves, A., Wayne, G., & Danihelka, I. (2014). Neural Turing Machines. arXiv preprint arXiv:1410.5401.