ElevenLabs has announced the launch of an AI dubbing model designed to capture emotional nuances across multiple languages.
Unlike traditional AI dubbing systems which rely on transcripts, Dubbing v2 conditions directly on the original performance, aiming to capture the speaker’s intonation and reproduce it across languages. The resulting output is “significantly more natural and expressive,” said ElevenLabs, while it remains faithful to the original delivery. Dubbing v2’s automated process provides natural dubbing in more than 90 languages.
The solution’s sync-aware translation system automatically aligns pacing, starts and stops, reducing the need for manual adjustment, adapting translations for spoken delivery while maintaining synchronisation with the original content. In what the company said is a first, Dubbing v2 makes translated speech feel like the original person actually said it.
Combining Dubbing v2 with professional localisation services including human translators, voice casting and professional audio mixing, ElevenProductions provides studios and broadcasters with a scalable workflow for professional grade multilingual productions.
Additionally, Dubbing v2 aims to enable creators to localise content such as YouTube videos with a single click inside ElevenCreative, allowing “significantly faster” global delivery without sacrificing quality.
Dubbing V2 is available in ElevenCreative and through ElevenProductions. API access is expected to be available soon.
