Memory-enhanced decoder for neural machine translation

Abstract
We propose to enhance the RNN decoder in a neural machine translator (NMT) with external memory, as a natural but powerful extension to the state in the decoding RNN. This memory-enhanced RNN decoder is called MemDec. At each time during decoding, MemDec will read from this memory and wr to this memory once, both with content-based addressing. Unlike the unbounded memory in previous work RNNsearch to store the representation of source sentence, the memory in MemDec is a matrix with pre-determined size designed to better capture the information important for the decoding process at each time step. Our empirical study on Chinese-English translation shows that it can improve by BLEU upon Groundhog and BLEU upon on Moses, yielding the best performance achieved with the same training set.

Please cite as:

@article{wang2016memory,
  title={Memory-enhanced decoder for neural machine translation},
  author={Wang, Mingxuan and Lu, Zhengdong and Li, Hang and Liu, Qun},
  journal={arXiv preprint arXiv:1606.02003},
  year={2016}
}