notesum.ai

Published at October 23

Lightweight Neural App Control

math.CO

cs.CC

cs.DM

Released Date: October 23, 2024

Authors: Filippos Christianos¹, Georgios Papoudakis¹, Thomas Coste¹, Jianye Hao¹, Jun Wang², Kun Shao¹

Aff.: ¹Huawei Noah's Ark Lab; ²University College London

Arxiv: https://arxiv.org/abs/2410.17883v1

Refer to caption

Model	Size $\downarrow$	Avg Inf. Time (s) $\downarrow$	Overall $\uparrow$
Model	Size $\downarrow$	Avg Inf. Time (s) $\downarrow$	AitW	AndCtrl
$\text{SeeAct}_{\text{choice}}$	unk	9.81	37.7	29.9
$\text{SeeAct}_{\text{ann}}$	unk	9.76	42.5	35.5
T3A	unk	4.87	26.9	53.1
M3A	unk	10.64	35.6	57.5
Florence2	820M	0.50	70.8	57.0
LiMAC with Florence2 (ours)	+520M	0.34	72.2	63.1
Qwen2-VL	2B	3.03	51.0	52.2
LiMAC with Qwen2-VL (ours)	+520M	0.63	70.9	62.5