VoiceMixer: Adversarial Voice Style Mixup

Although recent advances in voice conversion have shown significant improvement, there still remains a gap between the converted voice and target voice. A key factor that maintains this gap is the insufficient decomposition of content and voice style from the source speech. This insufficiency leads to the converted speech containing source speech style or losing source speech content. In this paper, we present VoiceMixer which can effectively decompose and transfer voice style through a novel information bottleneck and adversarial feedback. With self-supervised representation learning, the proposed information bottleneck can decompose the content and style with only a small loss of content information. Also, for adversarial feedback of each information, the discriminator is decomposed into content and style discriminator with self-supervision, which enable our model to achieve better generalization to the voice style of the converted speech. The experimental results show the superiority of our model in disentanglement and transfer performance, and improve audio quality by preserving content information.

We compare VoiceMixer with several VC models as:

1. StarGAN-VC: StarGAN-based voice conversion model [Demo link]

2. AGAIN-VC: Voice conversion model using Activation Guidance and Adaptive Instance Normalization [Demo link]

3. AUTOVC: Auto-encoder based voice conversion model. We train AUTOVC in various settings with each having different sizes of information bottleneck and adversarial speaker classifier [Demo link]

4. Blow: Flow-based voice conversion model using hyper-conditioning [Demo link]

5. IDE-VC (Not implemented): Autoencoder-based voice conversion with information-theoretic disentanglement. There is no official source code for IDE-VC model. Please check the official demo page for comparision [Demo link]