Kentaro Mitsui(The University of Tokyo), Tomoki Koriyama(The University of Tokyo) and Hiroshi Saruwatari(The University of Tokyo)
Abstract:
Multi-speaker speech synthesis is a technique for modeling
multiple speakers' voices
with a single model.
Although many approaches using deep neural networks (DNNs) have been proposed,
DNNs are prone to overfitting
when the amount of training data is limited.
We propose a framework for multi-speaker speech synthesis
using deep Gaussian processes (DGPs);
a DGP is a deep architecture of Bayesian kernel regressions
and thus robust to overfitting.
In this framework, speaker information is fed to duration/acoustic models
using speaker codes.
We also examine the use of deep Gaussian process latent variable models (DGPLVMs).
In this approach,
the representation of each speaker is learned simultaneously with other model parameters,
and therefore the similarity or dissimilarity of speakers
is considered efficiently.
We experimentally evaluated two situations to investigate the effectiveness of
the proposed methods.
In one situation,
the amount of data from each speaker is balanced (speaker-balanced),
and in the other, the data from certain speakers are limited (speaker-imbalanced).
Subjective and objective evaluation results showed that
both the DGP and DGPLVM synthesize multi-speaker speech more effective
than a DNN in the speaker-balanced situation.
We also found that the DGPLVM outperforms the DGP significantly in the speaker-imbalanced situation.