notesum.ai

Published at December 9

Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesis

cs.CV

Released Date: December 9, 2024

Authors: M. Hamza Mughal¹, Rishabh Dabral¹, Merel C. J. Scholman, Vera Demberg², Christian Theobalt²

Aff.: ¹Max Planck Institute for Informatics; ²Max Planck Institute for Informatics, Saarland University

Arxiv: http://arxiv.org/pdf/2412.06786v1

[Uncaptioned image]

	1 Speaker					All Speakers
	FID $\downarrow$	BeatAlign $\rightarrow$	L1Div $\rightarrow$	Diversity $\rightarrow$		FID $\downarrow$	BeatAlign $\rightarrow$	L1Div $\rightarrow$	Diversity $\rightarrow$
GT		$0.703$	$11.97$	$127$			$0.477$	$7.29$	$110$
CaMN [39]	$0.604$	$0.711$	$9.97$	$107$		$0.512$	$0.200$	$5.58$	$98$
EMAGE [40]	$0.570$	$0.793$	$11.41$	$124$		$0.692$	$0.284$	$6.06$	$88$
Audio2Photoreal [51]	$1.02$	$0.550$	$12.47$	$145$		$0.849$	$0.326$	$6.24$	$99$
ReMoDiffuse [68]	$0.702$	$0.824$	$12.46$	$123$		$1.120$	$0.218$	$5.06$	$116$
Ours (w/ Discourse)	$0.879$	$0.730$	$12.62$	$129$		$0.447$	$0.417$	$9.03$	$114$
Ours (w/ LLM & Gesture Type)	$0.808$	$0.734$	$11.97$	$122$		$0.487$	$0.514$	$9.94$	$118$