notesum.ai

Published at November 26

FTMoMamba: Motion Generation with Frequency and Text State Space Models

cs.CV

Released Date: November 26, 2024

Authors: Chengjian Li¹, Xiangbo Shu¹, Qiongjie Cui¹, Yazhou Yao¹, Jinhui Tang¹

Aff.: ¹School of Computer Science and Engineering, Nanjing University of Science and Technology

Arxiv: http://arxiv.org/abs/2411.17532v1

Refer to caption

	FID $\downarrow$	R-Precision $\uparrow$
Methods	FID $\downarrow$	Top-1	Top-2	Top-3	MM-Dist $\downarrow$	Diversity $\uparrow$	MModality $\uparrow$
Real motions	$0.002^{\pm.000}$	$0.511^{\pm.003}$	$0.703^{\pm.003}$	$0.797^{\pm.002}$	$2.974^{\pm.008}$	$9.503^{\pm.065}$	-
Hier [8]	$6.532^{\pm.024}$	$0.301^{\pm.002}$	$0.425^{\pm.002}$	$0.552^{\pm.004}$	$5.012^{\pm.018}$	$8.332^{\pm.042}$	-
TEMOS [23]	$3.734^{\pm.028}$	$0.424^{\pm.002}$	$0.612^{\pm.002}$	$0.722^{\pm.002}$	$3.703^{\pm.008}$	$8.973^{\pm.071}$	$0.368^{\pm.018}$
TM2T[13]	$1.501^{\pm.017}$	$0.424^{\pm.003}$	$0.618^{\pm.003}$	$0.729^{\pm.002}$	$3.467^{\pm.011}$	$8.589^{\pm.076}$	$2.424^{\pm.093}$
T2M [12]	$1.067^{\pm.002}$	$0.457^{\pm.002}$	$0.639^{\pm.003}$	$0.740^{\pm.003}$	$3.340^{\pm.008}$	$9.188^{\pm.002}$	$2.090^{\pm.083}$
MDM [28]	${0.544}^{\pm.044}$	$0.320^{\pm.005}$	$0.498^{\pm.004}$	$0.611^{\pm.007}$	$5.566^{\pm.027}$	$9.559^{\pm.086}$	$\mathbf{2.799}^{\pm.072}$
MotionDiffuse [35]	$0.630^{\pm.001}$	${0.491}^{\pm.001}$	${0.681}^{\pm.001}$	${0.782}^{\pm.001}$	${3.113}^{\pm.001}$	${9.410}^{\pm.049}$	$1.553^{\pm.042}$
$\text{MLD}^{*}$ [5]	$0.421^{\pm.010}$	$0.468^{\pm.002}$	$0.658^{\pm.002}$	$0.759^{\pm.002}$	$3.267^{\pm.009}$	$9.701^{\pm.072}$	$2.600^{\pm.106}$
Motion Mamba [36]	${0.281}^{\pm.009}$	$\mathbf{0.502}^{\pm.003}$	$\mathbf{0.693}^{\pm.002}$	$\mathbf{0.792}^{\pm.002}$	$\mathbf{3.060}^{\pm.058}$	$\mathbf{9.871}^{\pm.084}$	$\underline{2.294}^{\pm.058}$
Fg-T2M [31]	$0.243^{\pm.019}$	$\underline{0.492}^{\pm.002}$	$\underline{0.683}^{\pm.003}$	$\underline{0.783}^{\pm.002}$	$3.109^{\pm.007}$	$9.278^{\pm.072}$	$1.614^{\pm.049}$
MotionGPT [17]	$\underline{0.232}^{\pm.008}$	$0.492^{\pm.003}$	$0.681^{\pm.003}$	$0.778^{\pm.002}$	$\underline{3.096}^{\pm.008}$	$9.528^{\pm.071}$	$2.008^{\pm.084}$
FTMoMamba(Ours)	$\mathbf{0.181}^{\pm.009}$	$0.489^{\pm.003}$	$0.680^{\pm.002}$	$0.777^{\pm.002}$	$3.151^{\pm.009}$	$\underline{9.789}^{\pm.085}$	$2.277^{\pm.099}$