Alzheimers Dementia Recognition through Spontaneous Speech (ADReSS)

Wed-SS-1-6-2 Disfluencies and Fine-Tuning Pre-trained Language Models for Detection of Alzheimer’s Disease

Jiahong Yuan(Baidu Research USA), Yuchen Bian(Baidu Research USA), Xingyu Cai(Baidu Research USA), Jiaji Huang(Baidu Research USA), Zheng Ye(Chinese Academy of Sciences) and Kenneth Church(Baidu Research USA)
Abstract: Disfluencies and language problems in Alzheimer’s Disease can be naturally modeled by fine-tuning Transformer-based pre-trained language models such as BERT and ERNIE. Using this method, we achieved 89.6% accuracy on the test set of the ADReSS (Alzheimer’s Dementia Recognition through Spontaneous Speech Challenge), a considerable improvement over the baseline of 75.0%, established by the organizers of the challenge. The best accuracy was obtained with ERNIE, plus an encoding of pauses. Robustness is a challenge for large models and small training sets. Ensemble over many runs of BERT/ERNIE fine-tuning reduced variance and improved accuracy. We found that um was used much less frequently in Alzheimer’s speech, compared to uh. We discussed this interesting finding from linguistic and cognitive perspectives.
Student Information

Student Events

Travel Grants