Wed-SS-1-6-2 Disfluencies and Fine-Tuning Pre-trained Language Models for Detection of Alzheimer’s Disease

Abstract: Disfluencies and language problems in Alzheimer’s Disease can be naturally modeled by fine-tuning Transformer-based pre-trained language models such as BERT and ERNIE. Using this method, we achieved 89.6% accuracy on the test set of the ADReSS (Alzheimer’s Dementia Recognition through Spontaneous Speech Challenge), a considerable improvement over the baseline of 75.0%, established by the organizers of the challenge. The best accuracy was obtained with ERNIE, plus an encoding of pauses. Robustness is a challenge for large models and small training sets. Ensemble over many runs of BERT/ERNIE fine-tuning reduced variance and improved accuracy. We found that um was used much less frequently in Alzheimer’s speech, compared to uh. We discussed this interesting finding from linguistic and cognitive perspectives.