Point and Control it!: Using Computer Vision for Service Discovery to Control Smart Objects

• Paulo Filipe Dantas UFC
• José Gilvan Rodrigues Maia UFC
• Windson Viana UFC

Resumo

The widespread of smart objects in our daily lives request the creation and analysis of new service discovery mechanisms and interaction techniques. In this work, we designed and evaluated a pointing-based interaction mechanism based on a Convolutional Neural Network classification method. We called it ViSCo (View, Scan, and Control it), which extends the openHAB service discovery mechanism of smart objets. ViSCo aggregates the users' field of view, captured by the camera of their smartphones, to reduce the service discovery results. 17 users evaluated the final solution remotely, in an environment with virtual devices. Participants used the ViSCo approach to find and control virtual devices by pointing to real objects in their homes (e.g., their TVs). System Usability Scale (SUS) survey about ViSCo results showed a good level of acceptance, with an average score of 83.97.

Palavras-chave: IoT, Smart Home, CNN, computer vision, object classification