notesum.ai

Published at November 25

Sonic: Shifting Focus to Global Audio Perception in Portrait Animation

cs.MM

cs.GR

cs.SD

eess.AS

Released Date: November 25, 2024

Authors: Xiaozhong Ji¹, Xiaobin Hu¹, Zhihong Xu², Junwei Zhu¹, Chuming Lin¹, Qingdong He¹, Jiangning Zhang¹, Donghao Luo¹, Yi Chen¹, Qin Lin¹, Qinglin Lu¹, Chengjie Wang¹

Aff.: ¹Tencent; ²Zhejiang University

Arxiv: http://arxiv.org/abs/2411.16331v1

Method / Metric	Lip sync	Motion diversity	ID consistency	Video Smoothness
Aniportrait	1.42	1.62	3.11	2.09
SadTalker	1.98	2.34	2.95	2.95
Echomimic	2.77	2.65	3.48	2.71
Hallo2	3.15	2.37	3.34	2.94
Sonic(Ours)	4.58 (45% $\uparrow$ )	4.55 (72% $\uparrow$ )	4.29 (23% $\uparrow$ )	4.66 (58% $\uparrow$ )