notesum.ai

Published at November 28

OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation

cs.AI

Released Date: November 28, 2024

Authors: Se-eun Yoon¹, Xiaokai Wei², Yexi Jiang², Rachit Pareek², Frank Ong², Kevin Gao², Julian McAuley¹, Michelle Gong²

Aff.: ¹University of California, San Diego; ²Roblox

Arxiv: http://arxiv.org/pdf/2411.19352v1

Refer to caption

	Method	Factuality	Relevance			Novelty		Coverage
	Method	Factual (↑)	Hit (↑)	Precision (↑)	Sim (↑)	Pop50 (↓)	RPop50 (↓)	Entropy (↑)	MaxFreq (↓)
	Pop	1.00 1.00	.08 .14	.02.04	.91 .89	1.00 1.00	10.31 7.97	5.61 5.64	0.15 .12
LLaMA-405B	Base LLM	.84 .88	.22 .23	.06 .06	.91 .88	.35 .48	3.60 3.84	7.16 6.57	.36 .53
	Base LLM + Div	.68 .70	.13 .16	.03 .04	.86 .84	.15 .17	1.52 1.39	7.66 7.63	.18 .27
	OMuleT w/ $\mathit{P}_{\mathit{LLM}}$	.98 .98	.22 18	.05 .05	.92 .89	.14 .17	1.39 1.35	8.97 9.18	.10 .12
	OMuleT w/ $\mathit{P}$	1.00 .99	.25 .23	.07 .07	.93 .89	.13 .21	1.38 1.63	8.81 8.85	.05 .16
GPT-4o	Base LLM	.90 .94	.26 .29	.07 .09	.90 .88	.42 .56	4.34 4.48	7.17 6.64	.20 .39
	Base LLM + Div	.59 .64	.16 .18	.04 .05	.73 .73	.11 .12	1.10 .96	8.15 8.53	.07 .10
	OMuleT w/ $\mathit{P}_{\mathit{LLM}}$	.98 .99	.22 .19	.06 .06	.93 .90	.16 .21	1.60 1.67	8.73 8.97	.10 .10
	OMuleT w/ $\mathit{P}$	.99 .99	.27 .24	.08 .08	.93 .89	.17 .27	1.71 2.14	8.68 8.71	.07 .12