notesum.ai
Published at December 9ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance
cs.CV
Released Date: December 9, 2024
Authors: Chunwei Wang1, Guansong Lu1, Junwei Yang1, Runhui Huang1, Jianhua Han1, Lu Hou1, Wei Zhang1, Hang Xu1
Aff.: 1Huawei Noah's Ark Lab

| Method | LLM | Num. of image-text pairs | Num. of interleaved data |
| Chameleon [47] | 7B from scratch | 1.4B | 400B tokens |
| LWM [30] | LLaMA-2-7B | 1B | - |
| Unified IO 2 [33] | 6.8B from scratch | 970M | 157M |
| SEED-LLaMA [15] | Vicuna-7B | 600M | 150M |
| AnyGPT [59] | LLaMA-2 7B | 300M | 7.3M |
| Janus [51] | DeepSeek-LLM-1.3B | 65M | - |
| ILLUME (Ours) | Vicuna-7B | 15 M | - |