Facial Detection in Uncontrolled Environments: Systematic Literature Review

  • Andre Ribeiro Moraes USP
  • Clodoaldo Aparecido de Moraes Lima USP


Facial detection is an extremely important topic in biometrics. In particular, its application in uncontrolled environments presents a great challenge for the various existing techniques. This work aims to present and evaluate recent studies on facial detection applied in uncontrolled environments through a systematic review of the literature that addresses what are the main techniques and bases of faces used, as well as what are the main uncontrolled scenarios explored in them. Were formulated three research questions and 85 papers were selected for analysis. Based on the analysis of these works, it can be concluded that the techniques based on convolutional neural networks are the most explored techniques in the literature and that the FDDB face database is the most used in the experiments and specific scenarios present in uncontrolled environments, as occlusion and variation of illumination, are still little investigated.
Palavras-chave: facial detection, uncontrolled environment, convolutional neural networks, systematic literature review


Mourad Ahmed, M Hassaballah, Yasser Salama Hassan, AH Abd-Ellah, and AS Abdel Rady. 2015. A Probabilistic Framework for Robust Face Detection.

Hesham A Alabbasi and Florica Moldoveanu. 2014. Human face detection from images, based on skin color. IEEE.

Yancheng Bai and Bernard Ghanem. 2017. Multi-scale fully convolutional network for face detection in the wild. IEEE.

Yancheng Bai, Yongqiang Zhang, Mingli Ding, and Bernard Ghanem. 2018. Finding tiny faces in the wild with generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 21–30.

Jeremiah R Barr, Kevin W Bowyer, and Patrick J Flynn. 2014. The effectiveness of face detection algorithms in unconstrained crowd scenes. IEEE.

Amir Benzaoui, Houcine Bourouba, and Abdelhani Boukrouche. 2012. System for automatic faces detection. IEEE.

Jorge Biolchini, Paula Gomes Mian, Ana Candida Cruz Natali, and Guilherme Horta Travassos. 2005. Systematic review in software engineering.

Eugene Borovikov, Szilárd Vajda, Michael Bonifant, and Michael Gill. 2018. Looking at faces in the wild.

Zhenhua Chai, Yu Zhang, Zhijun Du, Dong Wang, and Heydi Méndez-Vázquez. 2014. Learning flexible block based local binary patterns for unconstrained face detection. IEEE.

Shonal Chaudhry and Rohitash Chandra. 2016. Unconstrained Face Detection from a Mobile Source Using Convolutional Neural Networks. Springer. 

Qiaosong Chen, Fahai Shen, Yuanyuan Ding, Panhao Gong, Ya Tao, and Jin Wang. 2018. Face Detection Using R-FCN Based Deformable Convolutional Networks. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 4165–4170.

Jingjing Deng and Xianghua Xie. 2017. Detect face in the wild using CNN cascade with feature aggregation at multi-resolution. IEEE.

Jingjing Deng and Xianghua Xie. 2017. Nested Shallow CNN-Cascade for Face Detection in the Wild. IEEE.

Ahmed El-Barkouky, Ham Rara, Aly Farag, and Phil Womble. 2012. Face detection at a distance using saliency maps. IEEE.

Ahmed El-Barkouky, Ahmed Shalaby, Ali Mahmoud, and Aly Farag. 2014. Selective part models for detecting partially occluded faces in the wild. IEEE.

Zhenyu Fang, Jinchang Ren, Stephen Marshall, Huimin Zhao, Zheng Wang, Kaizhu Huang, and Bing Xiao. 2020. Triple loss for hard face detection. Neurocomputing (2020).

Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. 2010. Object detection with discriminatively trained part-based models. 

Christophe Garcia and Manolis Delakis. 2002. A neural architecture for fast and robust face detection. IEEE.

Deepak Ghimire and Joonwhoan Lee. 2013. A robust face detection method based on skin color and edges.

Palak Girdhar, Deepali Virmani, and S Saravana Kumar. 2019. A hybrid fuzzy framework for face detection and recognition using behavioral traits. Journal of Statistics and Management Systems 22, 2 (2019), 271–287.

Shazia Gul and Humera Farooq. 2015. A machine learning approach to detect occluded faces in unconstrained crowd scene. IEEE.

Abdenour Hadid and Matti Pietikainen. 2006. A hybrid approach to face detection under unconstrained environments. IEEE. 

Youssef Hbali, Lahoucine Ballihi, Younes Ed-doughmi, and Mohammed Sadgal. 2019. Learning Deep Features For Efficient Face Detection. In 2019 Third International Conference on Intelligent Computing in Data Sciences (ICDS). IEEE, 1–8.

Erik Hjelmås and Boon Kee Low. 2001. Face detection: A survey. 

Sheen Hsieh, Kuo-Chin Fan, and Chiunhsiun Lin. 2002. A statistic approach to the detection of human faces in color nature scene.

Deng-Yuan Huang, Chao-Ho Chen, Tsong-Yi Chen, Wu-Chih Hu, Zhi-Bin Guo, and Cheng-Kang Wen. 2020. High-efficiency face detection and tracking method for numerous pedestrians through face candidate generation. Multimedia Tools and Applications(2020), 1–26.

S Janahiram, Abeer Alsadoon, PWC Prasad, AMS Rahma, A Elchouemi, and SMN Arosha Senanayake. 2016. Detecting Occluded Faces in Unconstrained Crowd Digital Pictures. IEEE.

Feng Jiang, Jie Zhang, Liping Yan, Yuanqing Xia, and Shiguang Shan. 2018. A Three-Category Face Detector with Contextual Information on Finding Tiny Faces. In 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 2680–2684.

Barbara Kitchenham. 2004. Procedures for performing systematic reviews.

Brendan F Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, and Anil K Jain. 2015. Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A.

Lian Hock Koh, Surendra Ranganath, and YV Venkatesh. 2002. An integrated automatic face detection and recognition system.

Ming-Yi Lei, Song-Zhi Su, Shao-Zi Li, and Guo-Rong Cai. 2016. An Efficient Training Strategy for Face Detector in Specific Scenes. IEEE.

Huaer Li, Sharifa Alghowinem, Sabrina Caldwell, and Tom Gedeon. 2019. Interpretation of Occluded Face Detection Using Convolutional Neural Network. In 2019 IEEE 23rd International Conference on Intelligent Engineering Systems (INES). IEEE, 000165–000170.

Junjie Li, Saleem Karmoshi, and Ming Zhu. 2017. Unconstrained face detection based on cascaded Convolutional Neural Networks in surveillance video. IEEE.

Jianshu Li, Luoqi Liu, Jianan Li, Jiashi Feng, Shuicheng Yan, and Terence Sim. 2017. Towards a Comprehensive Face Detector in the Wild. 

Jianchao Li, Dongping Zhang, Kun Zhang, Kui Hu, and Li Yang. 2017. Real-time face detection during the night. IEEE.

Zhihang Li, Xu Tang, Xiang Wu, Jingtuo Liu, and Ran He. 2019. Progressively refined face detection through semantics-enriched representation learning. IEEE Transactions on Information Forensics and Security 15 (2019), 1394–1406.

Shengcai Liao, Anil K Jain, and Stan Z Li. 2016. A fast and accurate unconstrained face detector. 

Shilun Lin and Fei Su. 2016. FCFD: Teach the machine to accomplish face detection step by step. IEEE.

Yuguang Liu and Martin D Levine. 2017. Multi-Path Region-Based Convolutional Neural Network for Accurate Detection of Unconstrained”” Hard Faces””.

Chuan-Chuan Low, Lee-Yeng Ong, and Voon-Chet Koo. 2019. Experimental Study on Multiple Face Detection with Depth and Skin Color. In 2019 IEEE 9th Symposium on Computer Applications & Industrial Electronics (ISCAIE). IEEE, 114–119.

Jiang-Jing Lv, You-Ji Feng, Xiang-Dong Zhou, and Xi Zhou. 2016. Face Detection Using Hierarchical Fully Convolutional Networks. Springer.

Kai Ma and Jezekiel Ben-Arie. 2012. Vector array based multi-view face detection with compound exemplars. IEEE. 

Huda Mady and Shadi MS Hilles. 2018. Face recognition and detection using Random forest and combination of LBP and HOG features. In 2018 International Conference on Smart Computing and Electronic Enterprise (ICSCEE). IEEE, 1–7.

Darijan Marčetić, Tomislav Hrkać, and Slobodan Ribarić. 2016. Two-stage cascade model for unconstrained face detection. IEEE.

A Annie Micheal and P Geetha. 2017. Multi-view face detection using Normalized Pixel Difference feature. IEEE.

Hazar Mliki, Sahar Dammak, and Emna Fendri. 2020. An improved multi-scale face detection using convolutional neural network. SIGNAL IMAGE AND VIDEO PROCESSING(2020).

Rosali Mohanty and MV Raghunadh. 2016. A new approach to face detection based on YCgCr color model and improved AdaBoost algorithm. IEEE.

Victor-Emil Neagoe and Mihai Neghină. 2010. Face detection using a dual cross-validation of chrominance/luminance channel decisions and decorrelation of the color space. World Scientific and Engineering Academy and Society (WSEAS). 

Dinh-Luan Nguyen, Vinh-Tiep Nguyen, Minh-Triet Tran, and Atsuo Yoshitaka. 2015. Adaptive wildnet face network for detecting face in the wild. International Society for Optics and Photonics.

Mathias A Onabid and Djimelyi Tsamene Charly. 2017. Enhancing gray scale images for face detection under unstable lighting condition.

Javier Orozco, Brais Martinez, and Maja Pantic. 2015. Empirical analysis of cascade deformable models for multi-view face detection. 

Margarita Osadchy, Yann Le Cun, and Matthew L Miller. 2007. Synergistic face detection and pose estimation with energy-based models. 

Lei Pang, Yue Ming, and Li Chao. 2018. F-DR Net: Face detection and recognition in one net. In 2018 14th IEEE International Conference on Signal Processing (ICSP). IEEE, 332–337.

Alfredo Petrosino and Giuseppe Salvi. 2010. A Rough Fuzzy Neural Based Approach to Face Detection.

Hongwei Qin, Junjie Yan, Xiu Li, and Xiaolin Hu. 2016. Joint training of cascaded cnn for face detection.

Rajeev Ranjan, Vishal M Patel, and Rama Chellappa. 2015. A deep pyramid deformable part model for face detection.

Rajeev Ranjan, Swami Sankaranarayanan, Ankan Bansal, Navaneeth Bodla, Jun-Cheng Chen, Vishal M Patel, Carlos D Castillo, and Rama Chellappa. 2018. Deep Learning for Understanding Faces: Machines May Be Just as Good, or Better, than Humans.

Shivkaran Ravidas and MA Ansari. 2019. Deep learning for pose-invariant face detection in unconstrained environment. International Journal of Electrical and Computer Engineering 9, 1(2019), 577.

Xuepeng Shi, Shiguang Shan, Meina Kan, Shuzhe Wu, and Xilin Chen. 2018. Real-time rotation-invariant face detection with progressive calibration networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2295–2303.

Han Shu, Dangdang Chen, Yali Li, and Shengjin Wang. 2017. A highly accurate facial region network for unconstrained face detection. IEEE.

Tioh Keat Soon, AH Basari, Nuzulha Khilwani Ibrahim, Burairah Hussin, Ariff Idris, Noorayisahbe Mohd Yaacob, Mustafa Almahdi Algaet, and Norazira A Jalil. 2018. The Utilization of Feature based Viola-Jones Method for Face Detection in Invariant Rotation. Int. J. Adv. Comput. Sci. Appl 9, 12 (2018), 73–78.

Chaowei Tang, Shiyu Chen, Xu Zhou, Shuai Ruan, and Haotian Wen. 2020. Small-Scale Face Detection Based on Improved R-FCN. Applied Sciences 10, 12 (2020), 4177.

Xu Tang, Daniel K Du, Zeqiang He, and Jingtuo Liu. 2018. Pyramidbox: A context-assisted single shot face detector. In Proceedings of the European Conference on Computer Vision (ECCV). 797–813.

Moeen Tayyab and MF Zafar. 2009. Face detection using 2D-discrete cosine transform and back propagation neural network. IEEE.

Danai Triantafyllidou, Paraskevi Nousi, and Anastasios Tefas. 2017. Fast deep convolutional face detection in the wild exploiting hard sample mining.

Danai Triantafyllidou, Paraskevi Nousi, and Anastasios Tefas. 2018. Fast deep convolutional face detection in the wild exploiting hard sample mining.

Ren Tsang, Joao Paulo Magalhaes, and George DC Cavalcanti. 2012. Combined AdaBoost and gradientfaces for face detection under illumination problems. IEEE.

Dzmitry Tsishkou, Mohamed Hammami, and Liming Chen. 2003. Face detection in video using combined data-mining and histogram based skin-color model. IEEE.

Michael Villamizar, Alberto Sanfeliu, and Francesc Moreno-Noguer. 2019. Online learning and detection of faces with low human supervision. The Visual Computer 35, 3 (2019), 349–370. 

Paul Viola and Michael Jones. 2001. Rapid object detection using a boosted cascade of simple features. IEEE.

Dong Wang, Jing Yang, Jiankang Deng, and Qingshan Liu. 2016. FaceHunter: A multi-task convolutional neural network based face detector. 

Jing-Wein Wang. 2006. Face Segmentation under Unconstrained Scenes. IEEE.

Shuainan Wang, Tong Xu, Wei Li, and Haifeng Sun. 2019. CSSD: Cascade Single Shot Face Detector. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.

Junjie Yan, Xucong Zhang, Zhen Lei, and Stan Z Li. 2013. Real-time high performance deformable model for face detection in the wild. Citeseer.

Bin Yang, Junjie Yan, Zhen Lei, and Stan Z Li. 2014. Aggregate channel features for multi-view face detection. IEEE.

Ming-Hsuan Yang, David J Kriegman, and Narendra Ahuja. 2002. Detecting faces in images: A survey. 

Shuo Yang, Ping Luo, Chen-Change Loy, and Xiaoou Tang. 2015. From facial parts responses to face detection: A deep learning approach. 

Shuo Yang, Ping Luo, Chen Change Loy, and Xiaoou Tang. 2018. Faceness-net: Face detection through deep facial part responses.

Siqi Yang, Arnold Wiliem, and Brian C Lovell. 2016. To face or not to face: Towards reducing false positive of face detection. IEEE.

Siqi Yang, Arnold Wiliem, and Brian C Lovell. 2018. It takes two to tango: Cascading off-the-shelf face detectors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 535–543.

Stefanos Zafeiriou, Cha Zhang, and Zhengyou Zhang. 2015. A survey on face detection in the wild: past, present and future. 

Roziati Zainuddin and Sinan A Naji. 2010. Multi-skin color clustering models for face detection. International Society for Optics and Photonics.

Zulhadi Zakaria, Shahrel Azmin Suandi, and Junita Mohamad-Saleh. 2018. Hierarchical Skin-AdaBoost-Neural Network (H-SKANN) for multi-face detection.

Dan Zeng, Han Liu, Fan Zhao, Shiming Ge, Wei Shen, and Zhijiang Zhang. 2019. Proposal pyramid networks for fast face detection. Information Sciences 495(2019), 136–149.

Xiaoxing Zeng, Xiaojiang Peng, Yali Wang, and Yu Qiao. 2020. Finding hard faces with better proposals and classifier. Machine Vision and Applications 31, 7 (2020), 1–15.

Chan Zhang, Tao Li, Song Guo, Ning Li, Yingqi Gao, and Kai Wang. 2019. Aggregation Connection Network For Tiny Face Detection. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.

Jialiang Zhang, Xiongwei Wu, Steven CH Hoi, and Jianke Zhu. 2020. Feature agglomeration networks for single stage face detection. Neurocomputing 380(2020), 180–189.

Nanhai Zhang, Jiajie Han, Jiani Hu, and Weihong Deng. 2015. Metric Learning Based False Positives Filtering for Face Detection. Springer.

Nanhai Zhang, Jiajie Han, Jiani Hu, and Weihong Deng. 2016. Locally rejected metric learning based false positives filtering for face detection. Springer.

Shifeng Zhang, Xiaobo Wang, Zhen Lei, and Stan Z Li. 2019. Faceboxes: A CPU real-time and accurate unconstrained face detector. Neurocomputing 364(2019), 297–309.

Shifeng Zhang, Xiangyu Zhu, Zhen Lei, Xiaobo Wang, Hailin Shi, and Stan Z Li. 2018. Detecting face with densely connected face proposal network.

Yongqiang Zhang, Mingli Ding, Yancheng Bai, and Bernard Ghanem. 2019. Detecting small faces in the wild based on generative adversarial network and contextual information. Pattern Recognition 94(2019), 74–86.

Yutong Zheng, Chenchen Zhu, Khoa Luu, Chandrasekhar Bhagavatula, T Hoang Ngan Le, and Marios Savvides. 2016. Towards a deep learning framework for unconstrained face detection. IEEE.

Zexun Zhou, Zhongshi He, Yuanyuan Jia, Jinglong Du, Lulu Wang, and Ziyu Chen. 2020. Context prior-based with residual learning for face detection: A deep convolutional encoder–decoder network. Signal Processing: Image Communication 88 (2020), 115948.

Chenchen Zhu, Yutong Zheng, Khoa Luu, and Marios Savvides. 2017. Cms-rcnn: contextual multi-scale region-based cnn for unconstrained face detection. In Deep Learning for Biometrics. Springer, 57–79.

Chenchen Zhu, Yutong Zheng, Khoa Luu, and Marios Savvides. 2018. Enhancing Interior and Exterior Deep Facial Features for Face Detection in the Wild. IEEE
Como Citar

Selecione um Formato
MORAES, Andre Ribeiro; LIMA, Clodoaldo Aparecido de Moraes. Facial Detection in Uncontrolled Environments: Systematic Literature Review. In: SIMPÓSIO BRASILEIRO DE SISTEMAS DE INFORMAÇÃO (SBSI), 17. , 2021, Uberlândia. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2021 .