Assessor Models for Predicting and Explaining Aleatoric Uncertainty in Classification - Lima, Pedro B. S.; Prudêncio, Ricardo B. C.; Lorena, Ana Carolina; Valeriano, Maria Gabriela (Anais do Encontro Nacional de Inteligência Artificial e Computacional (ENIAC); 2024: Anais do XXI Encontro Nacional de Inteligência Artificial e Computacional, 2024)
Machine Learning (ML) models have been successfully adopted in several applications. However, despite their success, high levels of uncertainty can be observed depending on the problem being solved. Specifically, aleatoric uncertainty in classification is related to the inherent randomness associated with predictor and target attributes, usually observed for instances close to an area of class overlap. The literature on ML has developed different methods for quantifying uncertainty. The current paper addresses the complementary task of explaining uncertainty. For this, we proposed using assessors, which are meta-models trained to predict the performance of a given base model on a problem of interest. In our work, we adapted the concept of assessors to predict the aleatoric uncertainty of instances, measured by the average class entropy across a pool of diverse base models. Once built, eXplainable AI (XAI) techniques are adopted to extract explanations from the assessor. Experiments were performed in two case studies, using both a simulated and a real dataset for predicting the severity of COVID-19 patients. A pool of 12 base models was adopted to measure aleatoric uncertainty, and Random Forest was adopted as the assessor. The explanations extracted from the assessor were useful to identify specific features that cause high uncertainty in the classification problems.