notesum.ai

Published at November 6

AutoGameUI: Constructing High-Fidelity Game UIs via Multimodal Learning and Interactive Web-Based Tool

cs.HC

cs.AI

Released Date: November 6, 2024

Authors: Zhongliang Tang¹, Mengchen Tan¹, Fei Xia¹, Qingrong Cheng¹, Hao Jiang¹, Yongxiang Zhang¹

Aff.: ¹Tencent TiMi L1 Studio, China

Arxiv: http://arxiv.org/abs/2411.03709v1

Refer to caption

Method	Train Set	Similarity	Precision $\uparrow$	Recall $\uparrow$	F1-score $\uparrow$	RMSE $\downarrow$	PER $\downarrow$
LayoutGMN (Patil et al., 2021)	RICO	N2N	91.4	60.4	62.5	1.66	1.52
LayoutGMN (Patil et al., 2021)	GameUI	N2N	92.7	61.8	65.2	1.56	0.81
LayoutTrans (Gupta et al., 2021)	RICO	N2N	76.0	54.3	42.1	3.84	1.75
LayoutTrans (Gupta et al., 2021)	GameUI	N2N	78.6	54.8	45.1	5.48	2.48
LayoutDM (Inoue et al., 2023)	RICO	N2N	80.4	55.2	47.2	3.24	4.35
LayoutDM (Inoue et al., 2023)	GameUI	N2N	81.6	55.5	48.6	4.00	4.46
Ours (w/o Reg)	RICO	N2N	93.4	63.7	66.8	1.63	0.94
	GameUI	N2N	94.2	64.1	68.9	1.65	1.05
	GameUI	N2G	96.0	68.0	74.4	1.16	0.58
Ours (w/i Reg)	GameUI	N2N	-	-	-	-	-
Ours (w/i Reg)	GameUI	N2G	96.2	68.7	75.2	0.78	0.36