Meta's New 'Movie Gen' AI System Can Deepfake Video From a Single Photo (arstechnica.com) 20
An anonymous reader quotes a report from Ars Technica: On Friday, Meta announced a preview of Movie Gen, a new suite of AI models designed to create and manipulate video, audio, and images, including creating a realistic video from a single photo of a person. The company claims the models outperform other video-synthesis models when evaluated by humans, pushing us closer to a future where anyone can synthesize a full video of any subject on demand. The company does not yet have plans of when or how it will release these capabilities to the public, but Meta says Movie Gen is a tool that may allow people to "enhance their inherent creativity" rather than replace human artists and animators. The company envisions future applications such as easily creating and editing "day in the life" videos for social media platforms or generating personalized animated birthday greetings.
Movie Gen builds on Meta's previous work in video synthesis, following 2022's Make-A-Scene video generator and the Emu image-synthesis model. Using text prompts for guidance, this latest system can generate custom videos with sounds for the first time, edit and insert changes into existing videos, and transform images of people into realistic personalized videos. [...] Movie Gen's video-generation model can create 1080p high-definition videos up to 16 seconds long at 16 frames per second from text descriptions or an image input. Meta claims the model can handle complex concepts like object motion, subject-object interactions, and camera movements. You can view example videos here. Meta also released a research paper with more technical information about the model.
As for the training data, the company says it trained these models on a combination of "licensed and publicly available datasets." Ars notes that this "very likely includes videos uploaded by Facebook and Instagram users over the years, although this is speculation based on Meta's current policies and previous behavior."
Movie Gen builds on Meta's previous work in video synthesis, following 2022's Make-A-Scene video generator and the Emu image-synthesis model. Using text prompts for guidance, this latest system can generate custom videos with sounds for the first time, edit and insert changes into existing videos, and transform images of people into realistic personalized videos. [...] Movie Gen's video-generation model can create 1080p high-definition videos up to 16 seconds long at 16 frames per second from text descriptions or an image input. Meta claims the model can handle complex concepts like object motion, subject-object interactions, and camera movements. You can view example videos here. Meta also released a research paper with more technical information about the model.
As for the training data, the company says it trained these models on a combination of "licensed and publicly available datasets." Ars notes that this "very likely includes videos uploaded by Facebook and Instagram users over the years, although this is speculation based on Meta's current policies and previous behavior."
Nothing is real anymore (Score:3)
Great. We have finally been able to get to the point where nothing you see online is real. It will all be AI generated. Congratulations to us. We have just destroyed the human race and given in to our robot overlords.
Re: (Score:3)
Re: (Score:2)
We're just moving the internet from being an historical non-fiction to being an absurdist semi-fiction, where there's just enough reality to make everything tangentially less real.
Yeah. Including humans.
and the plot's no problem (Score:1)
because people don't expect movies to have an interesting plot anymore.
What a gift! (Score:3)
What a wonderful gift to vindictive ex-spouses and divorce attorneys everywhere! Who wouldn't believe your ex cheated on you, why, when we have the video right here?
Just because you can do something doesn't mean you should. There have always been ways of faking photographs, etc..., but those generally required advanced skills and substantial amounts of time and money. This makes it possible for anyone with a grudge and a lawyer to use the legal system to punish anyone they dislike.
Re:What a gift! (Score:4, Insightful)
Re: (Score:3)
I think civil lawsuits will not be the major problem, because
1) videos produced by Meta models will be watermarked wityh their own Stable Signature https://ai.meta.com/blog/stabl... [meta.com] or a future equivalent. so if you're hit with a fraudulent filing, your attorney will suggest an expertise, which will trivially reveal the fraud. This will happen naturally as in a year or two everybody will know videos can be faked and expertise are required.
2) Nobody other than the very big actors (Meta, OpenAI, Adobe, cinema
One word (Score:1)
Porn
Is this, um, widely available? (Score:2)
My friends will be shocked to find out how long I've been dating Emma Watson.
Pearl Clutching (Score:2)
All the comments so far seem to assume this is the end of the human race (one literally claimed it). People also did the same "preacher's wife in The Simpsons" bit when photo and audio editing came out.
But neither Photoshop nor Pro Tools have led to the end of trust in photos/audio. You might be a little more skeptical of things that seem fake ... but otherwise life goes on. The same thing is true here: we've had video editing for years now and society hasn't collapsed.
Meanwhile, an entire generation of
Re: (Score:2)
"But neither Photoshop nor Pro Tools have led to the end of trust in photos/audio."
Because they are not content generators, they are content editors, and no made the arguments for them you claim they did.
"Meanwhile, an entire generation of incredible film makers from all over the world ... kids who can barely afford a computer,..."
Sure, kids who cannot afford a computer are "incredible film makers" that will make the next "Apocalypse Now" despite being unable to afford access to the technology that will ena
2D and 3D (Score:3)
Current AI doesn't really model in 3D, but rather apes patterns it sees in 2D frames. There's lot of subtle things this approach gets wrong in terms of consistency and perspective.
I suspect future AI will build a 3D model of the scene, not unlike for gaming, then map the movement of objects in model, then re-render it as 2D frames. It'll still make stupid shit sometimes, but at least the 3D aspects will "add up".
Re: (Score:1)
Addendum: making a 3D model first also allows human editors to more easily tweak it.
Re: (Score:2)
Current AI doesn't really model in 3D, but rather apes patterns it sees in 2D frames. There's lot of subtle things this approach gets wrong in terms of consistency and perspective.
One of the ways to spot a deepfaked image of a person is to look at the eyes. Normally there will be a light spot in the same position on each eye, which is a reflection from whatever is lighting the scene. Image generators don't/can't take this into account so you'll often see the light spots in different places.
Re: (Score:2)
Or the dancing 'ghost' (in a bedsheet) twirling around and around, under "Generate videos from text."
The inconsistency issue is real and probably will never be perfect, but if it's good enough...
The potential upside... (Score:2)
to this could be amazing. Being it's Meta, they will probably shit the bed but the technology itself could be nothing short of amazing.
Imagine being able to feed your book and some pictures to represent various characters and creatures into this and it pops out a series for you. It would be incredible. It won't need to come up with a story or script, as you provide that. It would need to "read the book" to interpret the scenes for the movie and you would want to train it on images that went with a specific
The Age of Aquarius was short lived. (Score:2)
Welcome to the dawning of the Age of Unreality.