notesum.ai
Published at November 15Generative AI in Multimodal User Interfaces: Trends, Challenges, and Cross-Platform Adaptability
cs.HC
cs.AI
Released Date: November 15, 2024
Authors: J. Bieniek, M. Rahouti, D. C. Verma
| Reference | Year | Focus | Multimodal Interaction | Generative AI | Cross-Platform | Challenges & Trends |
|---|---|---|---|---|---|---|
| Bieniek et al. (This Paper) | 2024 | A review of integrating Generative AI in modern computer UIs, focusing on multimodal interaction, cross-platform compatibility, and dynamic personalization | ✓ | ✓ | ✓ | ✓ |
| Jones et al. [2] | 2024 | Examines how multimodal LLMs ground language in comparison to human interactions | ✓ | ✗ | ✗ | ✱ |
| Munikoti et al. [3] | 2024 | Reviews architectures, challenges, and opportunities in multimodal AI, including transformer-based architectures for fusion | ✓ | ✓ | ✱ | ✓ |
| Huang et al. [4] | 2024 | Focuses on adaptive user experience with Generative AI and dynamic personalization | ✗ | ✓ | ✓ | ✱ |
| Kim et al. [5] | 2021 | Reviews multimodal interaction systems based on IoT and augmented reality | ✓ | ✗ | ✓ | ✓ |
| Lu et al. [6] | 2024 | Reviews AI in user experience design, with an emphasis on human-centered AI applications | ✗ | ✓ | ✓ | ✗ |
| Pyarelal et al. [7] | 2018 | Discusses automating the design of UIs using AI technologies | ✱ | ✓ | ✱ | ✱ |
| Su et al. [8] | 2023 | Reviews advancements in multimodal human-robot interaction, with emphasis on combining voice, gestures, and facial recognition | ✓ | ✗ | ✗ | ✓ |
| Zhang et al. [9] | 2024 | Evaluates the impact of multimodal interactions on user engagement in AI-driven conversations | ✓ | ✓ | ✗ | ✓ |
| Bandi et al. [10] | 2023 | Reviews generative AI models, input-output formats, evaluation metrics, and technical challenges | ✗ | ✓ | ✱ | ✓ |