notesum.ai

Published at November 22

OminiControl: Minimal and Universal Control for Diffusion Transformer

cs.CV

cs.AI

Released Date: November 22, 2024

Authors: Zhenxiong Tan¹, Songhua Liu¹, Xingyi Yang¹, Qiaochu Xue¹, Xinchao Wang¹

Aff.: ¹National University of Singapore

Arxiv: http://arxiv.org/abs/2411.15098v2

Refer to caption

Condition	Model	Method	Controllability	General Quality				Text Consistency
Condition	Model	Method	$\text{F1}\uparrow/\text{MSE}\downarrow$	$\text{FID}\downarrow$	$\text{SSIM}\uparrow$	$\text{MAN-IQA}\uparrow$	$\text{MUSIQ}\uparrow$	$\text{CLIP-Score}\uparrow$
Canny	SD1.5	ControlNet	0.34	18.74	0.35	0.45	67.81	0.75
	SD1.5	T2I-Adapter	0.22	20.06	0.35	0.39	67.88	0.74
	FLUX.1	ControlNet	0.21	98.68	0.25	0.37	56.90	0.53
	FLUX.1	Ours	0.38	20.63	0.40	0.61	75.91	0.76
Depth	SD1.5	ControlNet	923	23.02	0.34	0.47	70.73	0.726
	SD1.5	T2I-Adapter	1560	24.72	0.27	0.39	69.99	0.72
	FLUX.1	ControlNet	2958	62.20	0.26	0.38	66.84	0.54
	FLUX.1	Ours	903	27.26	0.39	0.55	75.06	0.728
Deblur	FLUX.1	ControlNet	572	30.38	0.74	0.31	54.37	0.78
Deblur	FLUX.1	Ours	132	11.49	0.87	0.39	67.63	0.87
Colorization	FLUX.1	ControlNet	351	16.27	0.64	0.43	70.95	0.85
Colorization	FLUX.1	Ours	24	10.23	0.73	0.43	70.74	0.90
Mask	SD1.5	ControlNet	7588	13.14	0.40	0.41	67.22	0.84
Mask	FLUX.1	Ours	6248	15.66	0.48	0.45	72.61	0.80