The external language models (LM) integration remains a challenging task for end-to-end
(E2E) automatic speech recognition (ASR) which has no clear division between acoustic
and language models. In this work, we propose an internal LM estimation (ILME) method to
facilitate a more effective integration of the external LM with all pre-existing E2E models with
no additional model training, including the most popular recurrent neural network transducer
(RNN-T) and attention-based encoder-decoder (AED) models. Trained with audio-transcript …

The efficacy of external language model (LM) integration with existing end-to-end (E2E)
automatic speech recognition (ASR) systems can be improved significantly using the
internal language model estimation (ILME) method [1]. In this method, the internal LM score
is subtracted from the score obtained by interpolating the E2E score with the external LM
score, during inference. To improve the ILME-based inference, we propose an internal LM
training (ILMT) method to minimize an additional internal LM loss by updating only the E2E …