notesum.ai

Published at October 30

MoLE: Enhancing Human-centric Text-to-image Diffusion via Mixture of Low-rank Experts

cs.CV

cs.AI

cs.LG

Released Date: October 30, 2024

Authors: Jie Zhu¹, Yixiong Chen², Mingyu Ding³, Ping Luo⁴, Leye Wang¹, Jingdong Wang⁵

Aff.: ¹Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, China and School of Computer Science, Peking University, Beijing, China; ²Johns Hopkins University; ³UC Berkeley; ⁴The University of Hong Kong; ⁵Baidu

Arxiv: http://arxiv.org/abs/2410.23332v1

Model	COCO Human Prompts
Model	HPS (%)	IR (%)
VQ-Diffision	$19.21\pm 0.04$	$-12.51\pm 2.44$
Versatile Diffusion	$19.75\pm 0.09$	$-8.81\pm 1.40$
SDXL	$20.84\pm 0.06$	$73.34\pm 2.29$
SD v1.5	$19.91\pm 0.09$	$28.34\pm 1.40$
MoLE (SD v1.5)	$20.27\pm 0.07$	$33.75\pm 1.49$
MoLE (SDXL)	$21.36\pm 0.02$	$98.52\pm 0.61$
Model	DiffusionDB Human Prompts
Model	HPS (%)	IR (%)
VQ-Diffision	$19.00\pm 0.02$	$-18.42\pm 1.49$
Versatile Diffusion	$20.09\pm 0.04$	$-29.05\pm 2.72$
SDXL	$21.51\pm 0.07$	$87.88\pm 2.53$
SD v1.5	$20.29\pm 0.01$	$-2.72\pm 1.66$
MoLE (SD v1.5)	$20.62\pm 0.04$	$4.36\pm 1.36$
MoLE (SDXL)	$22.35\pm 0.01$	$105.25\pm 1.15$