Synbody: Synthetic dataset with layered human models for 3d human perception and modeling

Z Yang, Z Cai, H Mei, S Liu, Z Chen… - Proceedings of the …, 2023 - openaccess.thecvf.com
Synthetic data has emerged as a promising source for 3D human research as it offers low-
cost access to large-scale human datasets. To advance the diversity and annotation quality …

DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance

Z Wang, J Jia, S Sun, H Wu, R Han… - Proceedings of the …, 2024 - openaccess.thecvf.com
Choreographers determine what the dances look like while cameramen determine the final
presentation of dances. Recently various methods and datasets have showcased the …

Cinematic Behavior Transfer via NeRF-based Differentiable Filming

X Jiang, A Rao, J Wang, D Lin… - Proceedings of the IEEE …, 2024 - openaccess.thecvf.com
In the evolving landscape of digital media and video production the precise manipulation
and reproduction of visual elements like camera movements and character actions are …

Embodied Human Activity Recognition

S Hu, Y Gong, G Mori - Proceedings of the IEEE/CVF Winter …, 2024 - openaccess.thecvf.com
We study how to utilize the mobility of an embodied agent to improve its ability to recognize
human activities. We introduce the embodied human activity recognition problem, where an …

UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Model

X Fan, J Li, Z Lin, W Xiao, L Yang - arXiv preprint arXiv:2408.00762, 2024 - arxiv.org
Audio-driven 3D facial animation aims to map input audio to realistic facial motion. Despite
significant progress, limitations arise from inconsistent 3D annotations, restricting previous …

CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion

Y Chen, A Rao, X Jiang, S Xiao, R Ma, Z Wang… - arXiv preprint arXiv …, 2024 - arxiv.org
With advancements in video generative AI models (eg, SORA), creators are increasingly
using these techniques to enhance video previsualization. However, they face challenges …

StoryDiffusion: How to Support UX Storyboarding With Generative-AI

Z Liang, X Zhang, K Ma, Z Liu, X Ren… - arXiv preprint arXiv …, 2024 - arxiv.org
Storyboarding is an established method for designing user experiences. Generative AI can
support this process by helping designers quickly create visual narratives. However, existing …

WHAC: World-grounded Humans and Cameras

W Yin, Z Cai, R Wang, F Wang, C Wei, H Mei… - arXiv preprint arXiv …, 2024 - arxiv.org
Estimating human and camera trajectories with accurate scale in the world coordinate
system from a monocular video is a highly desirable yet challenging and ill-posed problem …

Storyboarding in extended reality: a comparison between two tabletop solutions

F Manuri, F De Pace, V Carrescia, S Piumatti, A Sanna - 2024 - researchsquare.com
Storyboards are one of the most effective methods to pre-visualize a story: a sequence of
illustrations or pictures portraying settings, characters, actions, and events. This graphical …

DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis

Z Wang, J Li, X Qin, S Sun, S Zhou, J Jia, J Luo - ACM Multimedia 2024 - openreview.net
Synthesizing camera movement from music and dance is highly challenging due to the
contradictions and complexities of dance cinematography. Unlike human movement, which …