notesum.ai

Published at November 29

MoTe: Learning Motion-Text Diffusion Model for Multiple Generation Tasks

cs.CV

cs.CL

cs.LG

Released Date: November 29, 2024

Authors: Yiming Wu¹, Wei Ji¹, Kecheng Zheng¹, Zicheng Wang¹, Dong Xu¹

Aff.: ¹Unknown

Arxiv: http://arxiv.org/pdf/2411.19786v1

Dataset	Methods	R-Precision $\uparrow$			FID $\downarrow$	MMDist $\downarrow$	MModality $\uparrow$
Dataset	Methods	Top1	Top2	Top3	FID $\downarrow$	MMDist $\downarrow$	MModality $\uparrow$
HumanML3D	TM2T [10]	$0.424^{\pm.003}$	$0.618^{\pm.003}$	$0.729^{\pm.002}$	$1.501^{\pm.017}$	$3.467^{\pm.011}$	$\underline{2.424}^{\pm.093}$
	MotionDiffuse [7]	$0.491^{\pm.001}$	$0.681^{\pm.001}$	$0.782^{\pm.001}$	$0.630^{\pm.001}$	$3.113^{\pm.001}$	$1.553^{\pm.042}$
	MDM [6]	$0.320^{\pm.005}$	$0.498^{\pm.004}$	$0.611^{\pm.007}$	$0.544^{\pm.044}$	$5.566^{\pm.027}$	$\mathbf{2.799}^{\pm.072}$
	MLD [8]	$0.481^{\pm.003}$	$0.673^{\pm.003}$	$0.772^{\pm.002}$	$0.473^{\pm.013}$	$3.196^{\pm.010}$	$2.413^{\pm.079}$
	Fg-T2M [18]	$0.492^{\pm 0.03}$	$0.683^{\pm.003}$	$0.783^{\pm.002}$	$0.243^{\pm.019}$	$3.109^{\pm.007}$	$1.614^{\pm.049}$
	MotionGPT [11]	$0.492^{\pm.003}$	$0.681^{\pm.003}$	$0.778^{\pm.002}$	$0.232^{\pm.008}$	$3.096^{\pm.008}$	$2.008^{\pm.084}$
	HuTuMotion [41]	$0.500^{\pm.003}$	$0.686^{\pm.002}$	$0.782^{\pm.002}$	$0.224^{\pm.006}$	$3.058^{\pm.009}$	$0.966^{\pm.046}$
	T2M-GPT [9]	$0.491^{\pm.003}$	$0.680^{\pm.003}$	$0.775^{\pm.002}$	$0.116^{\pm.004}$	$3.118^{\pm.011}$	$1.856^{\pm.011}$
	ReMoDiffuse [42]	$0.510^{\pm.005}$	$0.698^{\pm.006}$	$0.795^{\pm.004}$	$0.103^{\pm.004}$	$2.974^{\pm.016}$	$1.795^{\pm.043}$
	DLP [43]	$0.495^{\pm.003}$	$0.651^{\pm.004}$	$0.792^{\pm.004}$	$\underline{0.071}^{\pm.002}$	$3.561^{\pm.017}$	$0.452^{\pm.012}$
	MoMask [44]	$\underline{0.521}^{\pm.002}$	$\underline{0.713}^{\pm.002}$	$\underline{0.807}^{\pm.002}$	$\mathbf{0.045}^{\pm.002}$	$\underline{2.958}^{\pm.008}$	$1.241^{\pm.040}$
	MoTe (Ours)	$\mathbf{0.548}^{\pm.002}$	$\mathbf{0.737}^{\pm.002}$	$\mathbf{0.825}^{\pm.002}$	$0.075^{\pm.004}$	$\mathbf{2.867}^{\pm.012}$	$2.399^{\pm.075}$
KIT	TM2T [10]	$0.280^{\pm.006}$	$0.463^{\pm.007}$	$0.587^{\pm.005}$	$3.599^{\pm.051}$	$4.591^{\pm.019}$	$\mathbf{3.292}^{\pm.034}$
	MotionDiffuse [7]	$0.417^{\pm.004}$	$0.621^{\pm.004}$	$0.739^{\pm.004}$	$1.954^{\pm.062}$	${2.958}^{\pm.005}$	$0.730^{\pm.013}$
	Fg-T2M [18]	${0.418}^{\pm.005}$	$0.626^{\pm.004}$	${0.745}^{\pm.004}$	$0.571^{\pm.047}$	$3.114^{\pm.015}$	$1.019^{\pm.029}$
	T2M-GPT [9]	$0.416^{\pm.006}$	$0.627^{\pm.006}$	${0.745}^{\pm.006}$	$0.514^{\pm.029}$	${3.007}^{\pm.023}$	$1.570^{\pm.039}$
	MotionGPT [11]	$0.366^{\pm.005}$	$0.558^{\pm.004}$	$0.680^{\pm.005}$	$0.510^{\pm.016}$	$3.527^{\pm.021}$	$2.328^{\pm.117}$
	MDM [6]	$0.164^{\pm.004}$	$0.291^{\pm.004}$	$0.396^{\pm.004}$	$0.497^{\pm.021}$	$9.191^{\pm.022}$	$1.907^{\pm.214}$
	MLD [8]	$0.390^{\pm.008}$	$0.609^{\pm.008}$	$0.734^{\pm.004}$	$0.404^{\pm.027}$	$3.204^{\pm.027}$	$2.192^{\pm.071}$
	MoMask [44]	$\mathbf{0.433}^{\pm.007}$	$\mathbf{0.656}^{\pm.005}$	$\mathbf{0.781}^{\pm.005}$	${0.204}^{\pm.011}$	$\mathbf{2.779}^{\pm.022}$	$1.131^{\pm.043}$
	HuTuMotion [41]	$0.409^{\pm.004}$	${0.640}^{\pm.004}$	$\underline{0.766}^{\pm.005}$	$\underline{0.201}^{\pm.064}$	$3.082^{\pm.025}$	$0.901^{\pm.035}$
	ReMoDiffuse [42]	$\underline{0.427}^{\pm.014}$	$\underline{0.641}^{\pm.004}$	${0.765}^{\pm.055}$	$\mathbf{0.155}^{\pm.006}$	$\underline{2.814}^{\pm.012}$	$1.239^{\pm.028}$
	MoTe (Ours)	$0.419^{\pm.005}$	${0.627}^{\pm.005}$	$0.741^{\pm.004}$	$0.256^{\pm.018}$	$3.216^{\pm.036}$	$\underline{2.615}^{\pm.103}$