notesum.ai

Published at November 13

Physics Informed Distillation for Diffusion Models

cs.LG

cs.AI

Released Date: November 13, 2024

Authors: Joshua Tian Jin Tee¹, Kang Zhang¹, Hee Suk Yoon¹, Dhananjaya Nagaraja Gowda², Chanwoo Kim³, Chang D. Yoo¹

Aff.: ¹School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST); ²Samsung Research; ³Department of Artificial Intelligence, Korea University

Arxiv: http://arxiv.org/abs/2411.08378v1

METHOD	Distance Metric	NFE	FID	IS
		( $\downarrow$ )	( $\downarrow$ )	( $\uparrow$ )
EDM (Karras et al., 2022)		36	2.04
EDM+Euler Solver (Karras et al., 2022)		250	2.10
DDPM Ho et al. (2020)		1000	3.17	9.46
DDIM (Song et al., 2021a)		10	13.36
		50	4.67
DPM-Solver-Fast (Lu et al., 2022)		10	4.70
3-DEIS (Zhang & Chen, 2023)		10	4.17
Teacher DDPM
Progressive Distillation (Salimans & Ho, 2022)	L2	1	8.34	8.69
DSNO* (Zheng et al., 2023)	LPIPS	1	3.78	-
Teacher Rectified Flow
2-Rectified Flow (+ distill)* (Liu et al., 2023)	LPIPS	1	4.85	9.01
Teacher EDM
Consistency Model (Song et al., 2023)	LPIPS	1	3.55	9.48
BOOT (Gu et al., 2023)	LPIPS	1	4.38	-
Diff-Instruct (Luo et al., 2023)	L2	1	4.12	9.89
Equilibrium Models (Geng et al., 2023)	L1	1	6.91	9.16
PID (Ours)	LPIPS	1	3.92	9.13