notesum.ai

Published at November 12

LAuReL: Learned Augmented Residual Layer

cs.LG

cs.AI

cs.CV

Released Date: November 12, 2024

Authors: Gaurav Menghani¹, Ravi Kumar¹, Sanjiv Kumar²

Aff.: ¹Institution 1; ²Institution 2

Arxiv: http://arxiv.org/abs/2411.07501v2

Model	Avg. Best	Params Added
	Accuracy@1	vs Baseline
	(%), 5 trials	(%)
Baseline	$74.95\pm 0.01$	-
Baseline + 1 Layer	$75.20\pm 0.12$	$4.37$
(Naive Scaling)
LAuReL-RW	$75.10\pm 0.10$	$0.003$
LAuReL-RW+LR	$75.20\pm 0.07$	$1.68$
LAuReL-RW+LR+PA	$75.25\pm 0.09$	$2.40$