notesum.ai

Published at May 3

AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising

NeurIPS

Released Date: May 3, 2024

Authors: Zigeng Chen¹, Xinyin Ma¹, Gongfan Fang¹, Zhenxiong Tan¹, Xinchao Wang¹

Aff.: ¹National University of Singapore

Arxiv: https://openreview.net/pdf/e39dedc1ff4ef521f777c210e981f87c12836a50.pdf

Refer to caption

Base Model	Configuration	Devices	MACs $\downarrow$	latency $\downarrow$	Speed up $\uparrow$	CLIP Score $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$
SD 2.1 (Text-to-Image)	Original Model	1	76T	5.51s	1.0x	31.60	27.89	–
	+ Ours (N=2 S=1)	2	38T	3.03s	1.8x	31.59	27.79	0.2121
	+ Ours (N=3 S=1)	3	25T	2.41s	2.3x	31.56	28.00	0.2755
	+ Ours (N=4 S=1)	4	19T	2.10s	2.6x	31.40	28.28	0.3132
	+ Ours (N=2 S=2)	3	19T	1.82s	3.0x	31.43	28.55	0.3458
	+ Ours (N=3 S=2)	4	13T	1.35s	4.0x	31.22	29.41	0.3778
SD 1.5 (Text-to-Image)	Original Model	1	34T	2.70s	1.0x	30.63	29.96	–
	+ Ours (N=2 S=1)	2	17T	1.52s	1.8x	30.62	29.94	0.1988
	+ Ours (N=3 S=1)	3	11T	1.23s	2.2x	30.58	29.87	0.2645
	+ Ours (N=4 S=1)	4	9T	1.01	2.6x	30.52	30.10	0.3073
	+ Ours (N=2 S=2)	3	9T	0.94s	2.9x	30.46	30.98	0.3232
	+ Ours (N=3 S=2)	4	6T	0.72s	3.7x	30.17	30.89	0.3811
SDXL (Text-to-Image)	Original Model	1	299T	13.81s	1.0x	32.33	27.43	–
	+ Ours (N=2 S=1)	2	150T	8.00s	1.7x	32.21	27.79	0.2509
	+ Ours (N=3 S=1)	3	100T	5.84s	2.4x	32.05	28.03	0.2940
	+ Ours (N=4 S=1)	4	75T	5.12s	2.7x	31.90	29.12	0.3157
	+ Ours (N=2 S=2)	3	75T	4.91s	2.8x	31.70	28.99	0.3209
	+ Ours (N=3 S=2)	4	49T	3.65s	3.8x	31.40	30.27	0.3556