Skip to yearly menu bar Skip to main content


Workshop

Stable and Effective Trainable Greedy Decoding for Sequence to Sequence Learning

Yun Chen · Kyunghyun Cho · Sam Bowman · Victor OK Li

East Meeting Level 8 + 15 #14

Mon 30 Apr, 11 a.m. PDT

We introduce a fast, general method to manipulate the behavior of the decoder in a sequence to sequence neural network model. We propose a small neural network actor that observes and manipulates the hidden state of a previously-trained decoder. We evaluate our model on the task of neural machine translation. In this task, we use beam search to decode sentences from the plain decoder for each training set input, rank them by BLEU score, and train the actor to encourage the decoder to generate the highest-BLEU output in a single greedy decoding operation without beam search. Experiments on several datasets and models show that our method yields substantial improvements in both translation quality and translation speed over its base system, with no additional data.

Live content is unavailable. Log in and register to view live content