notesum.ai

Published at October 30

Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot

cs.CL

cs.AI

Released Date: October 30, 2024

Authors: Sejin Lee¹, Dongha Kim², Min Song²

Aff.: ¹Yonsei University, dept. Library and Information Science; ²Onoma AI

Arxiv: http://arxiv.org/abs/2410.22767v1

Refer to caption

Joint Goal Accuracy (JGA)
	LLaMA3 + Instruction			GPT-3.5			GPT-4o
	MW2.0	MW2.4	SGD	MW2.0	MW2.4	SGD	MW2.0	MW2.4	SGD
CoT	0.4059	0.5182	0.7100	0.3931	0.4239	0.7092	0.4132	0.4837	0.7142
CoT + Persona	0.4258	0.5664	0.7819	0.4225	0.5247	0.7218	0.4319	0.5345	0.7438
SELF-DISCOVER	0.3932	0.4845	0.7539	0.3325	0.5326	0.6529	0.3499	0.4943	0.6954
ToT	0.2649	0.3616	0.7690	0.2974	0.3670	0.6296	0.2643	0.2834	0.6809
Slot F1
CoT	0.3455	0.3868	0.7151	0.3005	0.2507	0.6766	0.2636	0.3670	0.6882
CoT + Persona	0.4413	0.4706	0.8002	0.3405	0.3411	0.7293	0.3021	0.3781	0.7293
SELF-DISCOVER	0.3224	0.3300	0.5976	0.3236	0.3587	0.4407	0.3429	0.3832	0.7958
ToT	0.3098	0.3154	0.7860	0.2481	0.3306	0.7844	0.2149	0.2929	0.8223
Slot Accuracy
CoT	0.8092	0.6837	0.5565	0.8361	0.8246	0.5113	0.8299	0.8097	0.5246
CoT + Persona	0.8344	0.7902	0.6669	0.8364	0.8334	0.5740	0.8122	0.8249	0.5740
SELF-DISCOVER	0.8757	0.8767	0.4261	0.8223	0.8278	0.4826	0.8244	0.7823	0.5531
ToT	0.7971	0.7981	0.6474	0.9037	0.9214	0.6453	0.8109	0.9021	0.6983