NTMs are similar to a LSTM network, but instead of the memory stored in hidden state, the memory is stored somewhere external. The controller is the actual neural network, the memory is a matrix with memory size and item size as its axis, the read heads that “looks up” memory data, and a write head that decides where to write.
See Jonathan Hui’s explanation of NTM on Medium