Parallax Engine: Head Controlled Motion Parallax Using Notebooks’ RGB Camera

  • Jarbas Jácome Universidade Federal de Pernambuco - UFPE
  • Arlindo Gomes Universidade Federal de Pernambuco - UFPE
  • Willams de Lima Costa Universidade Federal de Pernambuco - UFPE
  • Lucas Silva Figueiredo Universidade Federal de Pernambuco - UFPE
  • Jader Abreu Universidade Federal de Pernambuco - UFPE
  • Luana Porciuncula Universidade Federal de Pernambuco - UFPE
  • Pedro K. Brant Universidade Federal de Pernambuco - UFPE
  • Luis E. M. Alves Universidade Federal de Pernambuco - UFPE
  • Walter F. M. Correia Universidade Federal de Pernambuco - UFPE
  • Veronica Teichrieb Universidade Federal de Pernambuco - UFPE
  • Jonysberg P. Quintino Universidade Federal de Pernambuco - UFPE
  • Fabio Q. B. da Silva Universidade Federal de Pernambuco - UFPE
  • Andre L. M. Santos Universidade Federal de Pernambuco - UFPE
  • Helder de Sousa Pinho SiDi

Resumo


Research on the Fish Tank Virtual Reality (FTVR) technique commonly uses specific sensors (e.g. infrared cameras and LEDs on glasses) to estimate user’s eye position. However, estimating the face position with an RGB camera is becoming more accessible. In this work, we explore community available face characteristics detection software to implement the FTVR technique for everyday uses of 3D-enabled applications on consumer notebooks without requiring extra devices. We introduce the Parallax Engine solution that can be added with ease to any Unity game engine application. The solution supports two parallax-related visualization options: 1) a monoscopic FTVR mode (FishTank), which locks the virtual camera of the 3D environment to the laptop’s screen 2) and a 2D parallax mode (Parallax2DoF), which allows horizontal and vertical displacement of 3D scene camera. Regarding face characteristics detection techniques, the Parallax Engine uses a standardized interface that can receive input from different methods and currently supports three options: Google’s MediaPipe, dlib, and PoseNet. We evaluated the proposed solution with five users, performing tasks using different options for viewing and face characteristics detection, aiming to understand how suitable it is for end-users. Besides some detection failures from dlib, results showed an overall good acceptance for both the FishTank and Parallax2DoF visualization options.

Palavras-chave: face characteristics detection, Fish tank virtual reality, motion parallax
Publicado
18/10/2021
JÁCOME, Jarbas et al. Parallax Engine: Head Controlled Motion Parallax Using Notebooks’ RGB Camera. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 23. , 2021, Evento Online. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2021 . p. 136-145.