Netflix has introduced a new tool that enables creatives to make changes to a scene without having to reshoot.
Video Object and Interaction Deletion (VOID), which the company has developed in collaboration with Sofia University, can delete an object from a scene and rebuild it as if the object never existed in the first place.
The tool is able to reimagine how everything else should behave once the object is removed.
According to the company, all a user needs to do is click on an object to remove it. A vision language model-based reasoning pipeline then identifies which other regions of the scene will be impacted, such as objects that might fall, collide or change trajectory, and encodes this into a “quadmask” that guides the diffusion model.
VOID’s first pass generates a video with the object and its interactions removed. If the model detects object morphing, it creates a second pass, rerunning inference using flow-warped noise derived from the first pass, stabilising object shape along the newly synthesised trajectories, said Netflix.
More details about VOID are available here.
