Video Audience Analysis using Bayesian Networks and Face Demographics
Resumo
In this paper, we propose an approach to study and to model audience attention to videos within digital signage scenarios. An experimental setup was conceived to simultaneously display videos of various categories and to capture videos from the audience and surrounding environment. Face analysis via a deep neural network is performed to estimate gender and age groups. In the proposed approach, a Bayesian Network is built to model possible relationships between the audience's age, gender and face size (which is indicative of the distance to the display) and the video content types. A publicly available video dataset of 152 videos was created for displaying purposes. An experimental evaluation indicated varying degrees of attention to different videos, depending on age and gender. The area under the ROC curve of the built Bayesian Network was 0.82. The proposed approach allows to better understand the possible relationships between audience demographics and video contents, which may, in turn, be useful for displaying the most appropriate content to a particular audience, help with the automatic insertion of ads (based on audience categories), among other applications.
Palavras-chave:
Digital Signage, Computer Vision, Audience Analysis, Bayesian Networks, Face Analysis
Referências
C. Palestini, "Digital heritage and earthquake emergency", Digital Heritage International Congress, 2013.
T. Kinebuchi, H. Arai, I. Miyagawa, S. Ando, K. Kataoka, H. Koike, "Image processing techniques for measuring advertising effectiveness of digital signage", R&D on Digital Signage as an Advertising Medium, 2009.
R. Ravnik, F. Solina, "Interactive and audience adaptive digital signage using real-time computer vision", International Journal of Advanced Robotic Systems, 2013.
P. D. S. Popai, Aplicações do digital signage, 2014.
R. Ravnik, F. Solina, "Audience measurement of digital signage: Quantitative study in real-world environment using computer vision", Interacting with Computers, 2013.
K. Mishima, T. Sakurada, Y. Hagiwara, "Low-cost managed digital signage system with signage device using small-sized and low-cost information device", IEEE Annual Consumer Communications & Networking Conference, Jan 2017.
R. Ravnik, F. Solina, "Interactive and audience-adaptive information interfaces", Electronic Lett. on Comp. Vision and Image Analysis, 2014.
K.-C. Yin, H.-C. Wang, D.-L. Yang, J. Wu, "A study on the effectiveness of digital signage advertisement", International Symposium on Computer Consumer and Control, 2012.
J. Grobelny, R. Michalski, "The role of background color interletter spacing and font size on preferences in the digital presentation of a product", Computers in Human Behavior, 2015.
Q. Feng, Y. Zheng, X. Zhang, Y. Song, Y. Jia Luo, Y. Li, T. Talhelm, "Gender differences in visual reflexive attention shifting: Evidence from an ERP study", Brain Research, 2011.
B. Moritz, Content in the train: Taking a closer look at in-train screens: A study on what types of content on in-train digital signage lead to a more positive customer experience, 2016.
M. Mikawa, S. Izumi, K. Tanaka, "Book recommendation signage system using silhouette-based gait classification", Int. Conf. on Machine Learning and Applications and Workshops, 2011.
A. L. Roggeveen, J. Nordfalt, D. Grewal, "Do Digital Displays Enhance Sales? Role of Retail Format and Message Content", Journal of Retailing, 2016.
V. Jain, E. Learned-Miller, "Fddb: A benchmark for face detection in unconstrained settings", University of Massachusetts Amherst Tech. Rep., 2010.
K. Ricanek, T. Tesafaye, "Morph: a longitudinal image database of normal adult age-progression", Int. Conf. on Automatic Face and Gesture Recognition, 2006.
N. Markus, M. Frljak, I. S. Pandzic, J. Ahlberg, R. Forchheimer, "A method for object detection based on pixel intensity comparisons", Computing Research Repository, 2013.
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. B. Girshick, S. Guadarrama, T. Darrell, "Caffe: Convolutional architecture for fast feature embedding", Computing Research Repository, 2014.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, "Going deeper with convolutions", Computing Research Repository, 2014.
I. de Pontes Oliveira, J. L. P. Medeiros, V. F. de Sousa, A. G. T. Júnior, E. T. Pereira, H. M. Gomes, "A data augmentation methodology to improve age estimation using convolutional neural networks", SIBGRAPI Conf. on Graphics Patterns and Images, 2016.
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, I. H. Witten, "The weka data mining software: An update", ACM SIGKDD Explorations Newsletter, 2009.
N. Howe, W. Strauss, Millennials Rising: The Next Great Generation, Knopf Doubleday Publishing Group, 2009.
N. Fitch, Sylvia Beach and the Lost Generation: A History of Literary Paris in the Twenties and Thirties, Norton, 1985.
J. D. Miller, "The generation x report: Active balanced and happy: These young americans are not bowling alone", Longitudinal study of American youth, 2011.
Resolução normativa, no. 2003, [online] Available: http://www.ans.gov.br/component/legislacao/?view=legislacao&task=TextoLei&format=raw&id=NzQ4.
G. F. Cooper, E. Herskovits, "A bayesian method for the induction of probabilistic networks from data", Machine Learning, 1992.
R. Daly, Q. Shen, S. Aitken, "Review: Learning bayesian networks: Approaches and issues", Knowledge Engineering Review, 2011.
N. Fenton, M. Neil, Risk Assessment and Decision Analysis with Bayesian Networks, CRC Press, 2012.
F. L. Valentim, Estudo e implementação de algoritmos de inferência e aprendizado em redes bayesianas, 2007.
G. F. Cooper, E. Herskovits, "A bayesian method for constructing bayesian belief networks from databases", Conf. on Uncertainty in Artificial Intelligence, 1991.
N. Friedman, D. Geiger, M. Goldszmidt, "Bayesian network classifiers", Machine Learning, 1997.
G. H. John, P. Langley, "Estimating continuous distributions in bayesian classifiers", Conf. on Uncertainty in Artific. Intell., 1995.
T. Fawcett, "An introduction to roc analysis", Pattern Recognition Letters, 2006.
M. Sokolova, G. Lapalme, "A systematic analysis of performance measures for classification tasks", Information Processing & Management, 2009.
Geraç ã o Global Estilos de Vida Nielsen, 20[online] Available: https://www.nielsen.com/br/pt/insights/reports/2015/Geracao-global-estilos-de-vida.html.
I. Inteligência, Sinais da nutrição, 2015.
K. Maeda, M. Nishi, T. Yoshida, K. Suzuki, H. Inoue, "Digital signage with audience detection using tv broadcasting waves", Int. Symp. on Applications and the Internet, 2010.
T. Kinebuchi, H. Arai, I. Miyagawa, S. Ando, K. Kataoka, H. Koike, "Image processing techniques for measuring advertising effectiveness of digital signage", R&D on Digital Signage as an Advertising Medium, 2009.
R. Ravnik, F. Solina, "Interactive and audience adaptive digital signage using real-time computer vision", International Journal of Advanced Robotic Systems, 2013.
P. D. S. Popai, Aplicações do digital signage, 2014.
R. Ravnik, F. Solina, "Audience measurement of digital signage: Quantitative study in real-world environment using computer vision", Interacting with Computers, 2013.
K. Mishima, T. Sakurada, Y. Hagiwara, "Low-cost managed digital signage system with signage device using small-sized and low-cost information device", IEEE Annual Consumer Communications & Networking Conference, Jan 2017.
R. Ravnik, F. Solina, "Interactive and audience-adaptive information interfaces", Electronic Lett. on Comp. Vision and Image Analysis, 2014.
K.-C. Yin, H.-C. Wang, D.-L. Yang, J. Wu, "A study on the effectiveness of digital signage advertisement", International Symposium on Computer Consumer and Control, 2012.
J. Grobelny, R. Michalski, "The role of background color interletter spacing and font size on preferences in the digital presentation of a product", Computers in Human Behavior, 2015.
Q. Feng, Y. Zheng, X. Zhang, Y. Song, Y. Jia Luo, Y. Li, T. Talhelm, "Gender differences in visual reflexive attention shifting: Evidence from an ERP study", Brain Research, 2011.
B. Moritz, Content in the train: Taking a closer look at in-train screens: A study on what types of content on in-train digital signage lead to a more positive customer experience, 2016.
M. Mikawa, S. Izumi, K. Tanaka, "Book recommendation signage system using silhouette-based gait classification", Int. Conf. on Machine Learning and Applications and Workshops, 2011.
A. L. Roggeveen, J. Nordfalt, D. Grewal, "Do Digital Displays Enhance Sales? Role of Retail Format and Message Content", Journal of Retailing, 2016.
V. Jain, E. Learned-Miller, "Fddb: A benchmark for face detection in unconstrained settings", University of Massachusetts Amherst Tech. Rep., 2010.
K. Ricanek, T. Tesafaye, "Morph: a longitudinal image database of normal adult age-progression", Int. Conf. on Automatic Face and Gesture Recognition, 2006.
N. Markus, M. Frljak, I. S. Pandzic, J. Ahlberg, R. Forchheimer, "A method for object detection based on pixel intensity comparisons", Computing Research Repository, 2013.
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. B. Girshick, S. Guadarrama, T. Darrell, "Caffe: Convolutional architecture for fast feature embedding", Computing Research Repository, 2014.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, "Going deeper with convolutions", Computing Research Repository, 2014.
I. de Pontes Oliveira, J. L. P. Medeiros, V. F. de Sousa, A. G. T. Júnior, E. T. Pereira, H. M. Gomes, "A data augmentation methodology to improve age estimation using convolutional neural networks", SIBGRAPI Conf. on Graphics Patterns and Images, 2016.
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, I. H. Witten, "The weka data mining software: An update", ACM SIGKDD Explorations Newsletter, 2009.
N. Howe, W. Strauss, Millennials Rising: The Next Great Generation, Knopf Doubleday Publishing Group, 2009.
N. Fitch, Sylvia Beach and the Lost Generation: A History of Literary Paris in the Twenties and Thirties, Norton, 1985.
J. D. Miller, "The generation x report: Active balanced and happy: These young americans are not bowling alone", Longitudinal study of American youth, 2011.
Resolução normativa, no. 2003, [online] Available: http://www.ans.gov.br/component/legislacao/?view=legislacao&task=TextoLei&format=raw&id=NzQ4.
G. F. Cooper, E. Herskovits, "A bayesian method for the induction of probabilistic networks from data", Machine Learning, 1992.
R. Daly, Q. Shen, S. Aitken, "Review: Learning bayesian networks: Approaches and issues", Knowledge Engineering Review, 2011.
N. Fenton, M. Neil, Risk Assessment and Decision Analysis with Bayesian Networks, CRC Press, 2012.
F. L. Valentim, Estudo e implementação de algoritmos de inferência e aprendizado em redes bayesianas, 2007.
G. F. Cooper, E. Herskovits, "A bayesian method for constructing bayesian belief networks from databases", Conf. on Uncertainty in Artificial Intelligence, 1991.
N. Friedman, D. Geiger, M. Goldszmidt, "Bayesian network classifiers", Machine Learning, 1997.
G. H. John, P. Langley, "Estimating continuous distributions in bayesian classifiers", Conf. on Uncertainty in Artific. Intell., 1995.
T. Fawcett, "An introduction to roc analysis", Pattern Recognition Letters, 2006.
M. Sokolova, G. Lapalme, "A systematic analysis of performance measures for classification tasks", Information Processing & Management, 2009.
Geraç ã o Global Estilos de Vida Nielsen, 20[online] Available: https://www.nielsen.com/br/pt/insights/reports/2015/Geracao-global-estilos-de-vida.html.
I. Inteligência, Sinais da nutrição, 2015.
K. Maeda, M. Nishi, T. Yoshida, K. Suzuki, H. Inoue, "Digital signage with audience detection using tv broadcasting waves", Int. Symp. on Applications and the Internet, 2010.
Publicado
28/10/2019
Como Citar
OLIVEIRA, ítalo de P. ; INTERAMINENSE, Carlos Daniel; PEREIRA, Eanes; GOMES, Herman M. .
Video Audience Analysis using Bayesian Networks and Face Demographics. In: CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 32. , 2019, Rio de Janeiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
DOI: https://doi.org/10.5753/sibgrapi.2019.9784.
