notesum.ai
Published at November 27FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion
cs.CV
Released Date: November 27, 2024
Authors: Haosen Yang1, Adrian Bulat1, Isma Hadji1, Hai X. Pham1, Xiatian Zhu2, Georgios Tzimiropoulos3, Brais Martinez1
Aff.: 1Samsung AI Center, Cambridge, UK; 2University of Surrey, UK; 3Queen Mary University, UK

| Method | Scaling Factor | FID | KID | FID | KID | CLIP | Latency(mins) |
|---|---|---|---|---|---|---|---|
| DemoFusion [3] | 63.24 | 0.0084 | 36.75 | 0.0096 | 32.0 | 2.5 | |
| AccDiffusion [15] | 59.42 | 0.0068 | 37.23 | 0.0105 | 31.69 | 2.6 | |
| FouriScale* [12] | 78.54 | 0.0136 | 40.80 | 0.0130 | 29.8 | 2.3 | |
| HiDiffusion [34] | 78.02 | 0.0136 | 51.41 | 0.0139 | 30.5 | 0.6 | |
| HiDiffusion [34] + FAM diffusion | 69.61 | 0.0140 | 34.26 | 0.0084 | 32.32 | 0.8 | |
| SDXL [19] | 59.47 | 0.0067 | 50.54 | 0.0136 | 30.6 | 0.8 | |
| SDXL [19] + FAM diffusion | 58.91 | 0.0072 | 33.96 | 0.0080 | 32.35 | 1 | |
| DemoFusion [3] | 68.82 | 0.0159 | 40.24 | 0.0122 | 32.0 | 8.6 | |
| AccDiffusion [15] | 73.47 | 0.0210 | 43.64 | 0.014 | 31.50 | 10 | |
| FouriScale* [12] | 73.57 | 0.0309 | 65.01 | 0.0357 | 28.54 | 6.2 | |
| HiDiffusion [34] | 112.51 | 0.0325 | 68.84 | 0.021 | 28.43 | 1.5 | |
| HiDiffusion [34] + FAM diffusion | 76.28 | 0.0007 | 36.70 | 0.010 | 32.26 | 1.8 | |
| SDXL [19] | 78.41 | 0.0136 | 69.40 | 0.0210 | 28.44 | 2.2 | |
| SDXL [19] + FAM diffusion | 69.25 | 0.0007 | 36.40 | 0.010 | 32.25 | 2.5 | |
| DemoFusion [3] | 65.89 | 0.0087 | 48.44 | 0.0157 | 30.45 | 19.6 | |
| AccDiffusion [15] | 73.97 | 0.0090 | 54.80 | 0.0187 | 30.15 | 20.5 | |
| FouriScale* [12] | 105.24 | 0.0342 | 70.45 | 0.0223 | 27.86 | 14.7 | |
| HiDiffusion [34] | 129.91 | 0.0483 | 156.98 | 0.0877 | 24.32 | 2.8 | |
| HiDiffusion [34] + FAM diffusion | 59.05 | 0.0074 | 44.65 | 0.0134 | 32.31 | 3.1 | |
| SDXL [19] | 160.10 | 0.0602 | 74.37 | 0.0242 | 26.70 | 5.4 | |
| SDXL [19] + FAM diffusion | 58.91 | 0.0073 | 43.65 | 0.0130 | 32.33 | 6.1 |