notesum.ai

Published at November 15

Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era

cs.CV

cs.AI

cs.HC

cs.LG

cs.MM

Released Date: November 15, 2024

Authors: Thanh Tam Nguyen¹, Zhao Ren², Trinh Pham³, Phi Le Nguyen⁴, Hongzhi Yin⁵, Quoc Viet Hung Nguyen¹

Aff.: ¹Griffith University; ²University of Bremen; ³Ho Chi Minh City University of Technology; ⁴Hanoi University of Science and Technology; ⁵The University of Queensland

Arxiv: http://arxiv.org/abs/2411.09955v1

Survey	Focused Task	Focused Modality	Key Contents
(Qin et al., 2024)	Editing	Text	Instruction development, Evaluation concerns
(Yin et al., 2023)	Editing	Text	LLM-empowered instructions, Instruction tuning
(Li et al., 2024b)	Retrieval	Image, Video, Audio	Image-text composite retrieval, Multimodal composite retrieval
(Zhan et al., 2023)	Generation	Image	Text guidance, Audio guidance, Sketch guidance, etc.
Ours	Editing	Image, Video, Audio	Instruction mechanisms, Augmentations, Learning stragies, Model designs, Loss functions