Meta announced its new tools, Emu Video and Emu Edit, that will allow users to edit images and generate videos for Facebook and Instagram using text prompts. These tools are built on Meta’s Emu, the firm’s first foundational model for image generation.
Meta, formerly known as Facebook, has unveiled its new tools enabling users to edit images and generate videos for Facebook and Instagram using text prompts.
The tools, Emu Video and Emu Edit are based on Meta’s Emu, the firm’s first foundational model for image generation.
Emu Video is a tool that allows users to create four-second-long videos using text prompts and reference images.
According to Meta, Emu Video leverages the firm’s Emu model with a text-to-video feature based on diffusion models, a generative model that can produce realistic images.
The video creation process involves two steps.
- First, users generate images using text prompts like “a cat wearing sunglasses”.
- Then, they create videos using the previously generated image alongside its corresponding caption, such as “the cat takes off the sunglasses and winks.
- Additionally, the tool can “animate” user-provided images based on a text prompt, such as “the Mona Lisa smiles and waves.”
Meta said that Emu Video can produce high-quality and faithful videos that human evaluators strongly prefer compared to prior work.
Emu Edit is a tool that offers users a user-friendly way to tweak images effortlessly. According to the firm, the tool “streamlines various image manipulation tasks and brings enhanced capabilities and precision to image editing.”
The tool allows users to manipulate the background of images, tweak the color and geometry of objects in the image, and perform many other functions using simple text commands, such as “change the sky to purple” or “make the car bigger.”
Meta said that Emu Edit can achieve this level of precision because it relies on a dataset that contains 10 million synthesized images, the largest of its kind. The tool also ensures that pixels in the input image unrelated to the instructions remain untouched.
Meta’s vision and plans
Meta said that its new tools will allow users to express themselves in new ways and explore the possibilities of image generation.
The tools are part of Meta’s vision to create a more immersive and interactive social media experience and advance the state-of-the-art in artificial intelligence.
Meta did not reveal when these tools would become publicly available for users.
Meta’s Emu model and tools are based on the research and development of the firm’s AI teams, such as Facebook AI Research (FAIR) and Facebook Reality Labs (FRL).
Meta has also collaborated with academic and industry partners, such as the University of California, Berkeley, and Adobe, to improve its image generation technology.
Meta previously announced its rebranding from Facebook in October as part of its shift to focus on building the metaverse, a virtual environment where people can interact with each other and digital content.
