notesum.ai

Published at November 25

Rethinking Diffusion for Text-Driven Human Motion Generation

cs.CV

Released Date: November 25, 2024

Authors: Zichong Meng¹, Yiming Xie¹, Xiaogang Peng¹, Zeyu Han¹, Huaizu Jiang¹

Aff.: ¹Northeastern University

Arxiv: http://arxiv.org/abs/2411.16575v1

	Methods	Framework	R-Precision $\uparrow$			FID $\downarrow$	Matching $\downarrow$	MModality $\uparrow$	CLIP-score $\uparrow$
	Methods	Framework	Top 1	Top 2	Top 3	FID $\downarrow$	Matching $\downarrow$	MModality $\uparrow$	CLIP-score $\uparrow$
HumanML3D	T2M-GPT [43]	VQ	$0.470^{\pm.003}$	$0.659^{\pm.002}$	$0.758^{\pm.002}$	$0.335^{\pm.003}$	$3.505^{\pm.017}$	$2.018^{\pm.053}$	$0.607^{\pm.005}$
	MMM [31]		$0.487^{\pm.003}$	$0.683^{\pm.002}$	$0.782^{\pm.001}$	$0.132^{\pm.004}$	$3.359^{\pm.009}$	$1.241^{\pm.073}$	$0.635^{\pm.003}$
	MoMask [13]		$0.490^{\pm.004}$	$0.687^{\pm.003}$	$0.786^{\pm.003}$	$0.116^{\pm.006}$	$3.353^{\pm.010}$	$1.263^{\pm.079}$	$\underline{0.637^{\pm.003}}$
	MDM-50Step [39]	Diffusion	$0.440^{\pm.007}$	$0.636^{\pm.006}$	$0.742^{\pm.004}$	$0.518^{\pm.032}$	$3.640^{\pm.028}$	$\mathbf{3.604^{\pm.031}}$	$0.578^{\pm.003}$
	MotionDiffuse [44]		$0.450^{\pm.006}$	$0.641^{\pm.005}$	$0.753^{\pm.005}$	$0.778^{\pm.005}$	$3.490^{\pm.023}$	$3.179^{\pm.046}$	$0.606^{\pm.004}$
	MLD [5]		$0.461^{\pm.004}$	$0.651^{\pm.004}$	$0.750^{\pm.003}$	$0.431^{\pm.014}$	$3.445^{\pm.019}$	$\underline{3.506^{\pm.031}}$	$0.610^{\pm.003}$
	ReMoDiffuse [45]		$0.468^{\pm.003}$	$0.653^{\pm.003}$	$0.754^{\pm.005}$	$0.883^{\pm.021}$	$3.414^{\pm.020}$	$2.703^{\pm.154}$	$0.621^{\pm.003}$
	Ours-DDPM	Autoregressive	$\underline{0.492^{\pm.006}}$	$\underline{0.690^{\pm.005}}$	$\underline{0.790^{\pm.005}}$	$\underline{0.116^{\pm.004}}$	$\underline{3.349^{\pm.010}}$	$2.470^{\pm.053}$	$0.637^{\pm.005}$
	Ours-SiT	Diffusion	$\mathbf{0.500^{\pm.004}}$	$\mathbf{0.695^{\pm.003}}$	$\mathbf{0.795^{\pm.003}}$	$\mathbf{0.114^{\pm.007}}$	$\mathbf{3.270^{\pm.009}}$	$2.231^{\pm.071}$	$\mathbf{0.642^{\pm.002}}$
KIT	T2M-GPT [43]	VQ	$0.359^{\pm.007}$	$0.553^{\pm.007}$	$0.690^{\pm.013}$	$0.593^{\pm.053}$	$3.765^{\pm.046}$	$1.798^{\pm.157}$	$0.651^{\pm.005}$
	MMM [31]		$0.363^{\pm.005}$	$0.569^{\pm.006}$	$0.724^{\pm.006}$	$0.478^{\pm.034}$	$3.629^{\pm.028}$	$1.455^{\pm.106}$	$0.660^{\pm.003}$
	MoMask [13]		$0.369^{\pm.005}$	$0.588^{\pm.005}$	$0.731^{\pm.005}$	$0.411^{\pm.026}$	$3.577^{\pm.021}$	$1.309^{\pm.058}$	$0.669^{\pm.002}$
	MDM [39]	Diffusion	$0.333^{\pm.012}$	$0.561^{\pm.009}$	$0.689^{\pm.009}$	$0.585^{\pm.043}$	$4.002^{\pm.033}$	$1.681^{\pm.107}$	$0.605^{\pm.007}$
	MotionDiffuse [44]		$0.344^{\pm.009}$	$0.536^{\pm.007}$	$0.658^{\pm.007}$	$3.845^{\pm.087}$	$4.167^{\pm.054}$	$1.774^{\pm.217}$	$0.626^{\pm.006}$
	MLD [5]		$0.351^{\pm.007}$	$0.536^{\pm.007}$	$0.658^{\pm.007}$	$0.492^{\pm.047}$	$3.746^{\pm.044}$	$\underline{1.803^{\pm.164}}$	$0.646^{\pm.006}$
	ReMoDiffuse [45]		$0.356^{\pm.004}$	$0.572^{\pm.007}$	$0.706^{\pm.009}$	$1.725^{\pm.053}$	$3.735^{\pm.036}$	$\mathbf{1.928^{\pm.127}}$	$0.665^{\pm.005}$
	Ours-DDPM	Autoregressive	$\underline{0.375^{\pm.006}}$	$\underline{0.597^{\pm.008}}$	$\underline{0.739^{\pm.006}}$	$\underline{0.340^{\pm.020}}$	$\underline{3.489^{\pm.018}}$	$1.479^{\pm.078}$	$\underline{0.681^{\pm.003}}$
	Ours-SiT	Diffusion	$\mathbf{0.387^{\pm.006}}$	$\mathbf{0.610^{\pm.006}}$	$\mathbf{0.749^{\pm.006}}$	$\mathbf{0.242^{\pm.014}}$	$\mathbf{3.374^{\pm.019}}$	$1.312^{\pm.053}$	$\mathbf{0.692^{\pm.002}}$