notesum.ai

Published at December 5

SocialMind: LLM-based Proactive AR Social Assistive System with Human-like Perception for In-situ Live Interactions

cs.AI

Released Date: December 5, 2024

Authors: Bufang Yang¹, Yunqi Guo¹, Lilin Xu², Zhenyu Yan¹, Hongkai Chen¹, Guoliang Xing¹, Xiaofan Jiang²

Aff.: ¹The Chinese University of Hong Kong, China; ²Columbia University, United States

Arxiv: http://arxiv.org/pdf/2412.04036v1

Refer to caption

Approach	Base LLM	Social Assistance	Multi Party Interactions	Multi-modal Sensor Data	Persona- lization	Interactive Mode	System Settings
Social-LLM (Jiang and Ferrara, 2023)	Sentence-BERT	○	○	○	○	Reactive	PC
Paprika (Jang et al., 2024)	GPT-4	●	○	○	○	Reactive	PC
Tianji (tia, 2024)	InternLM	●	○	○	○	Reactive	PC
Hua et. al (Hua et al., 2024b)	GPT 3.5, Atom-7B-Chat	●	●	○	○	Reactive	PC
SADAS (Hua et al., 2024a)	ChatGPT	●	●	○	○	Reactive	HoloLens
OS-1 (Xu et al., 2024)	GPT4, Gemini, Llama2	○	○	●	●	Proactive	Glasses
PRELUDE (Gao et al., 2024c)	GPT-4	○	○	○	●	Reactive	PC
SocialMind	GPT-4o, Llama-3.1	●	●	●	●	Proactive	Glasses