Multilingual Speech Recognition is a costly AI problem, as each language and even different accents require their own acoustic models to obtain best recognition performance.  Even though they all use the same phoneme symbols, each language and accent imposes its own coloring or “twang”.  In this talk, we propose a novel approach that uses a large multilingual model that is modulated  by the codes generated by an ancillary network that learns to code useful differences between the “twangs” or human languages.

This lecture is part of the closing day presentations for the 2018 Frederick Jelinek Memorial Summer Workshop.