Image Conductor: Precision Control for Interactive Video Synthesis

Y Li, X Wang, Z Zhang, Z Wang, Z Yuan, L Xie… - arXiv preprint arXiv …, 2024 - arxiv.org
Filmmaking and animation production often require sophisticated techniques for
coordinating camera transitions and object movements, typically involving labor-intensive …

Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics

R Li, C Zheng, C Rupprecht, A Vedaldi - arXiv preprint arXiv:2408.04631, 2024 - arxiv.org
We present Puppet-Master, an interactive video generative model that can serve as a motion
prior for part-level dynamics. At test time, given a single image and a sparse set of motion …

EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation

C Wang, J Gu, P Hu, H Zhao, Y Guo, J Han… - arXiv preprint arXiv …, 2024 - arxiv.org
Following the advancements in text-guided image generation technology exemplified by
Stable Diffusion, video generation is gaining increased attention in the academic …

ReVideo: Remake a Video with Motion and Content Control

C Mou, M Cao, X Wang, Z Zhang, Y Shan… - arXiv preprint arXiv …, 2024 - arxiv.org
Despite significant advancements in video generation and editing using diffusion models,
achieving accurate and localized video editing remains a substantial challenge …

This&That: Language-Gesture Controlled Video Generation for Robot Planning

B Wang, N Sridhar, C Feng, M Van der Merwe… - arXiv preprint arXiv …, 2024 - arxiv.org
We propose a robot learning method for communicating, planning, and executing a wide
range of tasks, dubbed This&That. We achieve robot planning for general tasks by …

Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions

A Taghipour, M Ghahremani, M Bennamoun… - arXiv preprint arXiv …, 2024 - arxiv.org
This paper investigates the role of CLIP image embeddings within the Stable Video Diffusion
(SVD) framework, focusing on their impact on video generation quality and computational …

TrackGo: A Flexible and Efficient Method for Controllable Video Generation

H Zhou, C Wang, R Nie, J Lin, D Yu, Q Yu… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent years have seen substantial progress in diffusion-based controllable video
generation. However, achieving precise control in complex scenarios, including fine-grained …