notesum.ai

Published at November 4

Fair In-Context Learning via Latent Concept Variables

cs.LG

cs.AI

cs.CL

Released Date: November 4, 2024

Authors: Karuna Bhaila¹, Minh-Hao Van¹, Kennedy Edemacu¹, Chen Zhao², Feng Chen³, Xintao Wu¹

Aff.: ¹University of Arkansas; ²Baylor University; ³University of Texas at Dallas

Arxiv: http://arxiv.org/abs/2411.02671v1

Refer to caption

External LLM	Method	Acc(%) $\uparrow$	F1(%) $\uparrow$	$\Delta$ SP $\downarrow$	$\Delta$ EO $\downarrow$
LLaMA-2-7B	Random Brown et al. (2020)	69.92_0.87	62.80_1.25	0.08_0.02	0.08_0.04
	LatentConcept Wang et al. (2024)	70.04_1.69	64.79_2.42	0.17_0.02	0.17_0.04
	Balanced Li et al. (2023b)	63.30_6.33	44.09_17.37	0.04_0.03	0.04_0.03
	Counterfactual Li et al. (2023b)	59.58_1.02	34.44_2.87	0.08_0.01	0.13_0.02
	Removal Li et al. (2023b)	64.60_6.25	47.92_17.34	0.09_0.05	0.11_0.06
	Instruction Li et al. (2023b)	63.86_6.35	46.64_14.86	0.07_0.06	0.09_0.05
	FairICL	68.48_0.89	64.42_1.01	0.02_0.03	0.01_0.04
LLaMA-2-13B	Random Brown et al. (2020)	76.00_1.19	75.75_1.44	0.14_0.04	0.11_0.08
	LatentConcept Wang et al. (2024)	77.48_0.70	77.22_0.74	0.16_0.02	0.12_0.01
	Balanced Li et al. (2023b)	74.58_1.38	71.92_2.19	0.13_0.05	0.10_0.07
	Counterfactual Li et al. (2023b)	68.18_2.05	57.39_4.55	0.13_0.06	0.17_0.08
	Removal Li et al. (2023b)	75.72_0.98	76.48_2.16	0.14_0.03	0.09_0.02
	Instruction Li et al. (2023b)	76.20_1.09	77.21_1.61	0.20_0.07	0.15_0.06
	FairICL	75.72_1.60	77.61_1.35	0.08_0.02	0.03_0.03