notesum.ai

Published at November 22

OminiControl: Minimal and Universal Control for Diffusion Transformer

cs.CV

cs.AI

Released Date: November 22, 2024

Authors: Zhenxiong Tan¹, Songhua Liu¹, Xingyi Yang¹, Qiaochu Xue¹, Xinchao Wang¹

Aff.: ¹National University of Singapore

Arxiv: http://arxiv.org/abs/2411.15098v1

[Uncaptioned image]

Methods	Base model	Parameters	Ratio
ControlNet	SD1.5 / 860M	361M	$\sim$ 42%
T2I-Adapter		77M	$\sim$ 9.0%
IP-Adapter		449M	$\sim$ 52.2%
ControlNet	FLUX.1 / 12B	3.3B	$\sim$ 27.5%
IP-Adapter	FLUX.1 / 12B	918M	$\sim$ 7.6%
Ours	FLUX.1 / 12B	14.5M / 48.7M w/ Encoder	$\sim$ 0.1% / $\sim$ 0.4% w/ Encoder