notesum.ai

Published at October 30

Why Gradient Subspace? Identifying and Mitigating LoRA's Bottlenecks in Federated Fine-Tuning of Large Language Models

cs.LG

cs.AI

Released Date: October 30, 2024

Authors: Navyansh Mahla¹, Ganesh Ramakrishnan¹

Aff.: ¹Unknown

Arxiv: http://arxiv.org/abs/2410.23111v2

Refer to caption

No of clients	Method	F1 Score
3	FedFTG	0.6064
	FlexLoRA	0.5638
	FFA-LoRA	0.0165
4	FedFTG	0.843
	FlexLoRA	0.4868
	FFA-LoRA	0.3175
5	FedFTG	0.7274
	FlexLoRA	0.6116
	FFA-LoRA	0.1104