notesum.ai

Published at December 10

Adaptive Weighting Push-SUM for Decentralized Optimization with Statistical Diversity

cs.DC

Released Date: December 10, 2024

Authors: Yiming Zhou¹, Yifei Cheng¹, Linli Xu¹, Enhong Chen¹

Aff.: ¹University of Science and Technology of China (USTC), Hefei, China

Arxiv: http://arxiv.org/pdf/2412.07252v1

Refer to caption

Model + Dataset		Resnet18 + Cifar10				Resnet50 + Cifar100				Average
Topology		Full	Divide	Exp	Random	Full	Divide	Exp	Random	R18 + C10	R50 + C100
SGP	Acc(%)	59.31	62.19	61.33	53.48	51.28	51.97	52.96	47.33	59.08	50.89
SGP	Loss(E-3)	14.29	13.68	13.97	16.24	24.29	23.21	23.43	25.24	14.55	24.04
MSGP	Acc(%)	63.39	62.72	63.26	55.38	36.03	44.03	47.69	38.69	61.19	41.61
MSGP	Loss(E-3)	14.75	15.52	15.56	17.48	45.73	36.49	28.09	42.34	15.83	38.16
SGAP	Acc(%)	62.97	63.25	62.36	61.09	56.10	57.06	56.93	49.93	62.42	55.00
SGAP	Loss(E-3)	13.27	12.51	13.20	13.44	22.18	20.74	21.68	24.24	13.11	22.21
MSGAP	Acc(%)	66.58	68.62	64.52	58.83	50.11	50.77	51.23	46.79	64.64	49.73
MSGAP	Loss(E-3)	12.85	11.33	15.29	16.89	28.08	33.87	26.90	32.11	14.09	30.24
S-ADDOPT	Acc(%)	60.20	59.47	59.76	54.36	54.24	53.97	55.62	31.36	58.45	48.80
S-ADDOPT	Loss(E-3)	13.76	14.35	13.91	14.96	22.58	23.18	22.78	27.36	14.25	23.98