Estimating Variable Importance and Interaction in Machine Learning via Genetic Algorithms - Guimarães, David Gabriel; Carvalho, Sabrina Sousa; Nascimento, Renato Higor do; Tinós, Renato (Anais da Brazilian Conference on Intelligent Systems (BRACIS); 2025: Anais da XXXV Brazilian Conference on Intelligent Systems, 2025)
This work explores the use of the Genetic Algorithm with Linkage Learning (GAwLL) for the feature selection problem. A notable byproduct of GAwLL application for feature selection is the generation of a variable interaction graph, offering insights into feature dependencies. To extend the original C++ implementation—which was limited to the K-nearest neighbors (KNN) algorithm—we introduce PyGAwLLfs, a Python-based framework designed to support feature selection for a wide range of machine learning models. Using PyGAwLLfs, we assess GAwLL performance across various models, including decision trees, random forests, and artificial neural networks. Furthermore, based on the internal mechanisms of GAwLL, we present a novel method for estimating variable importance in machine learning. Experimental evaluation on multiple datasets, including one from particle physics, demonstrates PyGAwLLfs effectiveness in the estimation of variable importance and interaction. We compare the variable importance scores generated by PyGAwLLfs with those produced by traditional approaches.