Recovery from Model Inconsistency in Multilingual Speech Recognition