Alibaba has unveiled Wan 2.1-VACE (Video All-in-one Creation
Wan2.1-VACE supports video generation with multi-modal inputs, including text, image, and video, and also includes video editing capabilities. The tool can be used for creating short social media videos, as well as TV and film post production and special effects processing.
Its editing features include referencing images or frames, video repainting, modifying selected areas of the video and spatio-temporal extension, said Alibaba.
According to the company, the tool enables users to generate video containing specific interacting subjects based on image samples as well as bring static images to life by adding natural movement effects. It also offers users advanced video repainting functions such as pose transfer, motion control, depth control, and recolourisation.
Wan2.1-VACE supports adding, modification or deletion to selective specific areas of a video without affecting the surroundings, added Alibaba. It also allows for the extension of video boundaries while filling in content to enrich the visual experience.
Alibaba is open-sourcing the Wan2.1-VACE model in two versions; a 14-billion(B)-parameter and a 1.3-billion(B)-parameter. The models are available to download for free on Hugging Face and GitHub, as well as Alibaba Cloud’s open-source community, ModelScope.
