Today we're sharing the next milestone in our Seamless...
AI at Meta@AIatMeta
29 views
Dec 01, 2023
~1 min read
Advertisement
1
Today we're sharing the next milestone in our Seamless Communication research — a new family of AI translation models that preserve expression and deliver near-real time streaming translations.
More on this new work ➡️ bit.ly/3uBZAYG
More on the individual models 🧵
More on this new work ➡️ bit.ly/3uBZAYG
More on the individual models 🧵
2:53
2
SeamlessM4T v2 is a foundational multilingual & multitask model for both speech & text. It's the successor to SeamlessM4T, demonstrating performance improvements across ASR, speech-to-speech, speech-to-text & text-to-speech tasks.
Evaluated across all tasks and languages through a collection of automatic metrics, it significantly outperformed previous state-of-the-art models.
Evaluated across all tasks and languages through a collection of automatic metrics, it significantly outperformed previous state-of-the-art models.
3
SeamlessExpressive enables the transfer of tones, emotional expression and vocal styles in speech translation. It incorporates an expressivity encoder and expressive unit-to-speech generator conditioned on source speech to deliver translations that maintain the unique nuances of the original speaker.
You can try a demo of SeamlessExpressive now using your own voice as an input ➡️ bit.ly/47Czovx
You can try a demo of SeamlessExpressive now using your own voice as an input ➡️ bit.ly/47Czovx
0:18
4
SeamlessStreaming is a new model that enables streaming speech-to-speech and speech-to-text translations with <2 seconds of latency. To deliver stronger results and adapt to differences in language structures, it intelligently decides when it has enough context to output the next translated segment. It does this through a learned read/write policy which enables stronger performance across many different language pairs.
5
Combining the strength of these three models, we're also introducing Seamless, a unified model that merges the quality and multilinguality of SeamlessM4T v2, the low latency of SeamlessStreaming and the expression preservation of Seamless expressive into one unified system.
6
In addition to these models, we're sharing new datasets, watermarking research & papers on this work ➡️ bit.ly/3GlMEZy
We believe open science is crucial to AI translation that benefits everyone & we look forward to continuing this work with the research community.
We believe open science is crucial to AI translation that benefits everyone & we look forward to continuing this work with the research community.