Wed-1-9-8 Advancing Multiple Instance Learning with Attention Modeling for Categorical Speech Emotion Classification

Shuiyang Mao(The Chinese University of Hong Kong), P. C. Ching(The Chinese University of Hong Kong), C.-C. Jay Kuo(University of Southern California) and Tan Lee(The Chinese University of Hong Kong)
Abstract: Categorical speech emotion recognition is typically performed as a sequence-to-label problem, i.e., to determine the discrete emotion label of the input utterance as a whole. One of the main challenges in practice is that most of the existing emotion corpora do not give ground truth labels for each segment; instead, we only have labels for whole utterances. To extract segment-level emotional information from such weakly labeled emotion corpora, we propose using multiple instance learning (MIL) to learn segment embeddings in a weakly supervised manner. Also, for a sufficiently long utterance, not all of the segments contain relevant emotional information. In this regard, three attention-based neural network models are then applied to the learned segment embeddings to attend the most salient part of a speech utterance. Experiments on the CASIA corpus and the IEMOCAP database show better or highly competitive results than other state-of-the-art approaches.
Student Information

Student Events

Travel Grants