notesum.ai

Published at December 10

Enhancing Relation Extraction via Supervised Rationale Verification and Feedback

cs.CL

cs.AI

Released Date: December 10, 2024

Authors: Yongqi Li¹, Xin Miao, Shen Zhou, Mayi Xu, Yuyang Ren, Tieyun Qian¹

Aff.: ¹Wuhan University, China

Arxiv: http://arxiv.org/pdf/2412.07289v1

Refer to caption

	Method	SemEval				TACRED				Re-TACRED				Avg.
	Method	5-shot	10-shot	20-shot	50-shot	5-shot	10-shot	20-shot	50-shot	5-shot	10-shot	20-shot	50-shot	Avg.
Random	In-context Learning	48.40	49.11	49.65	49.31	24.17	23.69	24.66	24.21	21.37	21.99	21.52	21.10	31.60
	w/ Self-Refine	47.93	48.50	49.19	48.88	23.15	23.05	24.18	23.12	21.04	21.51	20.71	21.32	31.05
	w/ Self-Consistency	49.30	49.09	50.11	50.35	25.69	24.76	25.64	25.42	22.08	22.56	21.84	22.01	32.40
	w/ GRACE	50.80	49.22	54.28	54.83	25.89	25.78	26.49	26.46	22.50	22.67	23.65	24.34	33.91
	w/ our SRVF	54.89	59.67	62.98	71.27	30.07	31.42	32.84	34.58	28.36	31.49	32.87	36.52	42.25
SimCSE	In-context Learning	57.33	59.13	62.49	64.26	27.48	28.64	30.08	27.81	34.78	41.85	42.82	43.69	43.36
	w/ Self-Refine	57.01	58.91	62.27	63.89	27.13	26.89	29.11	27.30	34.33	41.87	42.30	43.16	42.85
	w/ Self-Consistency	57.54	58.81	62.98	65.00	28.82	29.85	30.98	25.42	35.83	42.84	43.71	44.59	43.86
	w/ GRACE	57.93	58.48	66.32	67.48	28.76	28.60	30.03	26.46	33.95	41.53	42.35	44.37	43.86
	w/ our SRVF	60.76	64.12	69.54	76.32	32.99	33.50	34.81	36.13	39.48	46.54	49.73	54.31	49.85
Task-specific	In-context Learning	58.68	64.90	65.67	77.32	26.11	26.35	31.15	33.35	42.75	45.53	52.89	56.22	48.41
	w/ Self-Refine	58.38	64.96	65.68	77.35	25.01	25.48	30.62	32.67	42.10	44.98	52.11	55.62	47.91
	w/ Self-Consistency	59.62	65.45	65.74	77.48	26.93	26.83	31.67	33.61	43.54	46.04	53.34	56.69	48.91
	w/ GRACE	60.83	65.14	66.21	76.98	27.12	26.34	30.95	33.40	43.12	45.23	52.61	55.83	48.65
	w/ our SRVF	62.12	67.03	68.94	80.08	30.50	30.92	34.83	36.32	46.13	48.09	55.07	59.82	51.65