AI-Driven Responsive GUI for XR: Context-Aware Visual Adaptation - Santos, André Corrêa; Barão, Pedro Bittar; Lima, Rafael Melhado Araujo; Soares, Luciano Pereira (Anais do Simpósio de Realidade Virtual e Aumentada (SVR), 2025)
Augmented Reality enhances a user’s perception of the real world by overlaying digital information onto the physical environment in real time. One common means of interaction within AR is through Graphical User Interfaces, however, these interfaces can inadvertently occlude important elements of the physical scene. Although window repositioning is straightforward in traditional 2D interfaces, spatial placement in 3D AR environments is more complex, as it must consider factors such as user position, viewing direction, and contextual relevance. This work proposes an approach for the automatic positioning and coloring of GUI elements in AR, enabling interactive panels to dynamically adapt to environmental and contextual conditions. The system adjusts interface placement and appearance in real time based on contextual cues, such as object detection, human presence, and lighting changes. To accomplish this, computer vision techniques are employed to identify and classify objects, while generative AI models interpret high-level contextual information. This enables the interface to reposition itself to avoid occlusions and adjust visual properties, such as color and contrast, for optimal legibility and user comfort. The design and refinement of the system were guided by iterative user testing to ensure usability and effectiveness.