notesum.ai

Published at November 5

ATM: Improving Model Merging by Alternating Tuning and Merging

cs.LG

cs.AI

cs.CV

Released Date: November 5, 2024

Authors: Luca Zhou¹, Daniele Solombrino¹, Donato Crisostomi¹, Maria Sofia Bucarelli¹, Fabrizio Silvestri¹, Emanuele Rodolà

Aff.: ¹Sapienza University of Rome

Arxiv: http://arxiv.org/abs/2411.03055v1

Refer to caption

	CIFAR100	DTD	EuroSAT	GTSRB	MNIST	RESISC45	SVHN	Average
Task Arithmetic	0.59	0.46	0.78	0.61	0.96	0.63	0.77	0.69
TIES	0.70	0.51	0.76	0.60	0.94	0.73	0.78	0.72
Breadcrumbs	0.60	0.47	0.77	0.60	0.95	0.63	0.75	0.68
ValFT ATM: 10 Orders	0.78	0.61	0.53	0.97	0.99	0.88	0.95	0.82
ATM: 10 Orders	0.79	0.61	0.98	0.97	0.99	0.89	0.96	0.89
ATM: 30 Orders	0.83	0.68	0.99	0.99	1.00	0.94	0.97	0.91