Meta's 'Massively Multilingual' AI Model Translates Up To 100 Languages, Speech or Text 14
An anonymous reader quotes a report from Ars Technica: On Tuesday, Meta announced SeamlessM4T, a multimodal AI model for speech and text translations. As a neural network that can process both text and audio, it can perform text-to-speech, speech-to-text, speech-to-speech, and text-to-text translations for "up to 100 languages," according to Meta. Its goal is to help people who speak different languages communicate with each other more effectively. Continuing Meta's relatively open approach to AI, Meta is releasing SeamlessM4T under a research license (CC BY-NC 4.0) that allows developers to build on the work. They're also releasing SeamlessAlign, which Meta calls "the biggest open multimodal translation dataset to date, totaling 270,000 hours of mined speech and text alignments." That will likely kick-start the training of future translation AI models from other researchers.
Among the features of SeamlessM4T touted on Meta's promotional blog, the company says that the model can perform speech recognition (you give it audio of speech, and it converts it to text), speech-to-text translation (it translates spoken audio to a different language in text), speech-to-speech translation (you feed it speech audio, and it outputs translated speech audio), text-to-text translation (similar to how Google Translate functions), and text-to-speech translation (feed it text and it will translate and speak it out in another language). Each of the text translation functions supports nearly 100 languages, and the speech output functions support about 36 output languages.
Among the features of SeamlessM4T touted on Meta's promotional blog, the company says that the model can perform speech recognition (you give it audio of speech, and it converts it to text), speech-to-text translation (it translates spoken audio to a different language in text), speech-to-speech translation (you feed it speech audio, and it outputs translated speech audio), text-to-text translation (similar to how Google Translate functions), and text-to-speech translation (feed it text and it will translate and speak it out in another language). Each of the text translation functions supports nearly 100 languages, and the speech output functions support about 36 output languages.
Accuracy? (Score:5, Funny)
My hovercraft is full of eels.
Re: (Score:3)
Believe it or not, I have high expectations here. Transformers are a good fit for translation tasks. It's a real shame that we don't have more quality training data, and that's only going to get more difficult now.
I haven't read the whitepaper yet, so I hope they have a plan to minimize the amount of their machine translated text that ends up in future datasets.
Re: (Score:2)
Re: (Score:2)
I've been using Google Translate since it started and I've found that it has improved dramatically over that time, but still struggles in certain situations.
It used to translate everything into the first person, which made headlines read a bit weirdly. They seem to have improved that, but it still struggles with lack of context. As an example, it often gets subject's gender confused, even when their name and/or gender is mentioned in the same sentence.
It also has trouble with slang and informal speech, at l
Ya, but unfortunately ... (Score:2)
Meta's 'Massively Multilingual' AI Model Translates Up To 100 Languages, Speech or Text
It only translates things into an incomprehensible dead language that Professor Farnsworth calls, "crazy gibberish" [youtube.com] ...
Historical data (Score:4, Funny)
Unfortunately, they trained on historical data sets, so when they translate "Out of sight, out of mind", the result comes out as "invisible idiot".
Subtitles (Score:3)
Re: (Score:3)
Re: (Score:2)
I've found it useful on Android for videos that don't have subtitles, like on news websites.
Re: (Score:2)
They cannot even get basic search to work. (Score:1)
Begs the question: (Score:1)
what's a "minimally multilingual AI model"?
full of mistakes (Score:2)
I asked for a translation into Navajo and got one into (bad? good?) Hopi, which is a totally unrelated language. I pointed out the mistake, and got one into something resembling Navajo except it was half English and half really badly put together Navajo forms.