notesum.ai

Published at December 6

KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning

cs.CL

cs.AI

Released Date: December 6, 2024

Authors: Peng Yu¹, Cheng Deng¹, Beiya Dai¹, Xinbing Wang¹, Ying Wen¹

Aff.: ¹Shanghai Jiao Tong University

Arxiv: http://arxiv.org/pdf/2412.04948v1

Refer to caption

structure-based methods
Method	WN18RR					FB15k-237
Method	MR	MRR	H@1	H@3	H@10	MR	MRR	H@1	H@3	H@10
TransE	2300	0.243	0.043	0.441	0.532	323	0.279	0.198	0.376	0.441
DistMult	7000	0.444	0.412	0.470	0.504	512	0.281	0.199	0.301	0.446
RotatE	3340	0.476	0.428	0.492	0.571	177	0.338	0.241	0.375	0.533
description-based methods (autoencoder PLMs)
StAR	51	0.401	0.243	0.491	0.709	117	0.296	0.205	0.322	0.482
C-LMKE	72	0.598	0.480	0.675	0.806	183	0.404	0.324	0.439	0.556
SimKGC	-	0.671	0.587	0.731	0.817	-	0.333	0.246	0.362	0.510
description-based methods (autoregressive LLMs)
Llama-2-7B	15969	0.010	0.004	0.010	0.020	5359	0.006	0.002	0.004	0.012
Llama2-7B_KaLM	19	0.556	0.409	0.656	0.851	114	0.299	0.204	0.325	0.502
Llama3-8B_KaLM	23	0.588	0.446	0.676	0.860	121	0.308	0.212	0.337	0.509
Mistral-7B_KaLM	20	0.612	0.484	0.702	0.869	116	0.317	0.225	0.351	0.518