Today we're sharing the next milestone in our Seamless Communication research — a new family of AI translation models that preserve expression and deliver near-real time streaming translations.
More on this new work ➡️ https://t.co/KNZCEEPk9v
More on the individual models 🧵 https://t.co/ZzCIR4GBe2
SeamlessM4T v2 is a foundational multilingual & multitask model for both speech & text. It's the successor to SeamlessM4T, demonstrating performance improvements across ASR, speech-to-speech, speech-to-text & text-to-speech tasks.
Evaluated across all tasks and languages through a collection of automatic metrics, it significantly outperformed previous state-of-the-art models.
Evaluated across all tasks and languages through a collection of automatic metrics, it significantly outperformed previous state-of-the-art models.
SeamlessExpressive enables the transfer of tones, emotional expression and vocal styles in speech translation. It incorporates an expressivity encoder and expressive unit-to-speech generator conditioned on source speech to deliver translations that maintain the unique nuances of the original speaker.
You can try a demo of SeamlessExpressive now using your own voice as an input ➡️ https://t.co/04ChruNiF3
You can try a demo of SeamlessExpressive now using your own voice as an input ➡️ https://t.co/04ChruNiF3
SeamlessStreaming is a new model that enables streaming speech-to-speech and speech-to-text translations with <2 seconds of latency. To deliver stronger results and adapt to differences in language structures, it intelligently decides when it has enough context to output the next translated segment. It does this through a learned read/write policy which enables stronger performance across many different language pairs.
Combining the strength of these three models, we're also introducing Seamless, a unified model that merges the quality and multilinguality of SeamlessM4T v2, the low latency of SeamlessStreaming and the expression preservation of Seamless expressive into one unified system. https://t.co/93PnlK7J3j
In addition to these models, we're sharing new datasets, watermarking research & papers on this work ➡️ https://t.co/GoJXI7ILV5
We believe open science is crucial to AI translation that benefits everyone & we look forward to continuing this work with the research community.
We believe open science is crucial to AI translation that benefits everyone & we look forward to continuing this work with the research community.
Generated by Thread Navigator
Press ⌘ + S to quick-export