Visual Theory of Mind for Human-Agent Collaboration in Smart Environments - Hoffmann, Sandy; Sarkadi, Stefan; Panisson, Alison R. (Anais do Simpósio Brasileiro de Sistemas Colaborativos (SBSC); 2026: Anais do XXI Simpósio Brasileiro de Sistemas Colaborativos, 2026)
Smart environments increasingly rely on intelligent agents to support people in daily activities, particularly in assisted living scenarios. For such systems to support collaboration with humans, agents must move beyond reactive perception and develop an explicit understanding of human states, intentions, and needs. This paper presents an approach for human-agent collaboration in smart environments based on Visual Theory of Mind (VToM), enabling an assistive agent to infer human intentions and contextual beliefs from visual inputs. These visual inferences are integrated with additional multimodal signals, including contextual and interaction-based inputs, to support proactive and adaptive assistance. By combining visual reasoning with multimodal information, the proposed approach allows the agent to align its behavior with human goals and ongoing activities, fostering effective collaboration rather than unilateral automation. We discuss the agent architecture, inference mechanisms, and collaborative interaction scenarios in smart homes and assisted living environments, highlighting how VToM contributes to human-centered assistance in shared environments.