notesum.ai

Published at December 5

SoRA: Singular Value Decomposed Low-Rank Adaptation for Domain Generalizable Representation Learning

cs.CV

Released Date: December 5, 2024

Authors: Seokju Yun¹, Seunghye Chae, Dongheon Lee, Youngmin Ro

Aff.: ¹Machine Intelligence Laboratory, University of Seoul, Korea

Arxiv: http://arxiv.org/pdf/2412.04077v1

Single-source DGSS Trained on GTAV
Synthetic-to-Real Generalization			Test Domains (mIoU in %)
Methods	Backbone	$\text{Params.}^{*}$	$\rightarrow$ Citys.	$\rightarrow$ BDD	$\rightarrow$ Map.	Avg.
$\mathbf{\circ}$ CLOUDS [3]	CLIP-CN-L	0.0M	60.20	57.40	67.00	61.50
$\mathbf{\circ}$ VLTSeg [36]	EVA02-L	304.2M	65.30	58.30	66.00	63.20
$\mathbf{\circ}$ Rein [80]	EVA02-L	3.0M	65.30	60.50	64.90	63.60
$\mathbf{\circ}$ FADA [4]	EVA02-L	11.7M	66.70	61.90	66.10	64.90
$\bullet$ tqdm [59]	EVA02-L	304.2M	68.88	59.18	70.10	66.05
$\mathbf{\circ}$ SoRA (Ours)	EVA02-L	5.1M	68.05	60.81	68.33	65.73
$\bullet$ SoRA (Ours)	EVA02-L	5.1M	69.94	62.48	68.33	66.92
$\mathbf{\circ}$ DoRA [52]	DINOv2-L	7.5M	66.12	59.31	67.07	64.17
$\mathbf{\circ}$ VPT [37]	DINOv2-L	3.7M	68.75	58.64	68.32	65.24
$\mathbf{\circ}$ SET [86]	DINOv2-L	6.1M	68.06	61.64	67.68	65.79
$\mathbf{\circ}$ FADA [4]	DINOv2-L	11.7M	68.23	61.94	68.09	66.09
$\mathbf{\circ}$ AdaptFormer [12]	DINOv2-L	6.3M	70.10	59.81	68.77	66.23
$\mathbf{\circ}$ SSF [51]	DINOv2-L	0.5M	68.97	61.30	68.77	66.35
$\mathbf{\circ}$ LoRA [33]	DINOv2-L	7.3M	70.13	60.13	70.42	66.89
$\bullet$ $\text{Rein}^{\dagger}$ [80]	DINOv2-L	3.0M	70.68	62.51	69.61	67.60
$\mathbf{\circ}$ SoRA (Ours)	DINOv2-L	4.9M	71.82	61.31	71.67	68.27
$\bullet$ SoRA (Ours)	DINOv2-L	4.9M	73.63	63.33	70.98	69.31
Multi-source DGSS Trained on GTAV + SYNTHIA
$\mathbf{\circ}$ $\text{Rein}^{\dagger}$ [80]	DINOv2-L	3.0M	72.17	61.53	70.69	68.13
$\mathbf{\circ}$ SoRA (Ours)	DINOv2-L	4.9M	73.16	61.90	72.73	69.26
$\bullet$ SoRA (Ours)	DINOv2-L	4.9M	74.85	63.59	73.92	70.79
Multi-source DGSS Trained on GTAV + SYNTHIA + UrbanSyn
$\mathbf{\circ}$ FFT	DINOv2-L	304.2M	75.90	60.93	72.80	69.88
$\mathbf{\circ}$ SoRA (Ours)	DINOv2-L	4.9M	77.33	62.78	74.93	71.68
$\mathbf{\circ}$ $\text{FFT}^{\ddagger}$	DINOv2-L	307.3M	77.06	61.81	75.09	71.32
$\mathbf{\circ}$ $\text{Rein}^{\ddagger}$ [80]	DINOv2-L	3.0M	78.42	62.20	74.49	71.70
$\mathbf{\circ}$ $\text{SoRA}^{\ddagger}$ (Ours)	DINOv2-L	4.9M	79.22	63.84	76.30	73.12
$\mathbf{\circ}$ Freeze	DINOv2-G	0.0M	76.08	61.98	72.23	70.10
$\mathbf{\circ}$ FFT	DINOv2-G	1.1B	76.90	61.69	73.53	70.71
$\mathbf{\circ}$ SoRA (Ours)	DINOv2-G	6.6M	78.39	63.75	75.16	72.43
$\bullet$ SoRA (Ours)	DINOv2-G	6.6M	80.37	65.67	76.18	74.07