Directional Light Vector Estimation From a Virtual AR Object and its 2D Shadow Mask - Yacovenco, Aleksander; Vieira, Marcelo Bernardes; Maciel, Luiz Maurílio Da Silva; Da Silva, Rodrigo Luis De Souza (Anais do Simpósio de Realidade Virtual e Aumentada (SVR); 2025: Anais do XXVII Simpósio de Realidade Virtual e Aumentada, 2025)
This work presents a novel method for inferring the main directional light source in a 3D scene, given only 2D inputs, namely a camera image and a rough shadow mask. A two stage algorithm is proposed, in which the first stage handles the inputs and makes an initial estimation for the light source. The second stage refines this first estimate to find a new vector assumed to be closer to the real directional light vector. Both stages are performed by using one virtual object rendered into the real scene. In the first stage, an external shadow estimator produces a coarse shadow for the virtual object, enabling the computation of an initial directional light vector. Also, it is possible to compare the coarse shadow with a virtual shadow computed from the object. In the second stage, we seek to maximize the intersection over union (IoU) between both shadows. We assume that the best directional light vector provides the best shadow matching. The experiments are made both in virtual and real environments, in scenes with different levels of control and known data. Results show that our method is capable of finding the 3D light vector from the 2D scene, enhancing the initial rough shadow input.