Reducing Instability in Synthetic Data Evaluation with a Super-Metric in MalDataGen - Silva, Anna Luiza Gomes da; Kreutz, Diego; Diniz, Angelo; Mansilha, Rodrigo; Fonseca, Celso Nobre da (Anais da Escola Regional de Redes de Computadores (ERRC); 2025: Anais da XXII Escola Regional de Redes de Computadores, 2025)
Evaluating the quality of synthetic data remains a persistent challenge in the Android malware domain due to instability and the lack of standardization among existing metrics. This work integrates into MalDataGen a Super-Metric that aggregates eight metrics across four fidelity dimensions, producing a single weighted score. Experiments involving ten generative models and five balanced datasets demonstrate that the Super-Metric is more stable and consistent than traditional metrics, exhibiting stronger correlations with the actual performance of classifiers.