Mon-3-7-6 Unsupervised Domain Adaptation Under Label Space Mismatch for Speech Classification

Abstract: Unsupervised domain adaptation (UDA) using adversarial learning has shown promise in adapting speech models from a labeled source domain to an unlabeled target domain. However, prior works make a strong assumption that the label spaces of source and target domains are identical, which can be easily violated in real-world conditions. We present AMLS, an end-to-end architecture that performs Adaptation under Mismatched Label Spaces by separating shared and private classes in each domain using two binary classifiers that are learned in the adaptation process. An evaluation on three speech adaptation tasks, namely gender, microphone, and dataset adaptation, shows that AMLS provides as much as 15% accuracy gain over state-of-the-art baselines. In sum, our contribution paves the way for applying UDA to speech models in unconstrained settings with no assumptions on the source and target label spaces.