Wed-1-5-8 Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition

ShiLiang Zhang(Alibaba Group), Zhifu Gao(Machine Intelligence Technology, Alibaba Group), Haoneng Luo(School of Computer Science, Northwestern Polytechnical University), Ming Lei(Machine Intelligence Technology, Alibaba Group), Jie Gao(Machine Intelligence Technology, Alibaba Group), Zhijie Yan(Machine Intelligence Technology, Alibaba Group) and lei xie(School of Computer Science, Northwestern Polytechnical University)
Abstract: Recently, streaming end-to-end automatic speech recognition (E2E-ASR) has gained more and more attention. Many efforts have been paid to turn the non-streaming attention-based E2E-ASR system into streaming architecture. In this work, we propose a novel online E2E-ASR system by using \emph{Streaming Chunk-Aware Multihead Attention} (SCAMA) and a latency control memory equipped self-attention network (LC-SAN-M). LC-SAN-M uses chunk-level input to control the latency of encoder. As to SCAMA, a jointly trained \emph{predictor} is used to control the output of encoder when feeding to decoder, which enables decoder to generate output in streaming manner. Experimental results on the open 170-hour AISHELL-1 and an industrial-level 20,000-hour Mandarin speech recognition tasks show that our approach can significantly outperform the MoChA-based baseline system under comparable setup. On the AISHELL-1 task, our proposed method achieves a character error rate (CER) of 7.39\%, to the best of our knowledge, which is the best published performance for online ASR.
Student Information

Student Events

Travel Grants