notesum.ai

Published at December 9

ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance

cs.CV

Released Date: December 9, 2024

Authors: Chunwei Wang¹, Guansong Lu¹, Junwei Yang¹, Runhui Huang¹, Jianhua Han¹, Lu Hou¹, Wei Zhang¹, Hang Xu¹

Aff.: ¹Huawei Noah's Ark Lab

Arxiv: http://arxiv.org/pdf/2412.06673v1

Refer to caption

Method	LLM	Num. of image-text pairs	Num. of interleaved data
Chameleon [47]	7B from scratch	1.4B	400B tokens
LWM [30]	LLaMA-2-7B	1B	-
Unified IO 2 [33]	6.8B from scratch	970M	157M
SEED-LLaMA [15]	Vicuna-7B	600M	150M
AnyGPT [59]	LLaMA-2 7B	300M	7.3M
Janus [51]	DeepSeek-LLM-1.3B	65M	-
ILLUME (Ours)	Vicuna-7B	15 M	-