AutoML Optimized by Genetic Algorithm and Item Response Theory
Resumo
Gerar modelos de aprendizado de máquina automaticamente segue sendo uma área de muita pesquisa recente e desafiadora. E ainda não há uma forma definitiva capaz de gerar resultados satisfatórios e que apresente uma complexidade baixa. Com isso, neste trabalho, são apresentados os resultados de uma metodologia proposta que utiliza os conceitos da Teoria de Resposta ao Item e Algoritmo Genético para criar um algoritmo de AutoML do tipo NAS que seja capaz de gerar uma Rede Neural competitiva com os modelos já conhecidos na literatura. Nos resultados obtidos, foi possível gerar um modelo competitivo se comparado ao AutoKeras, porém com complexidade inferior a 5.5% da complexidade total do modelo do próprio AutoKeras.
Palavras-chave:
AutoML, Teoria de Resposta ao Item, Algoritmo Genético, Rede Neural
Referências
Baker, F. B. (2001). The basics of item response theory. ERIC.
Cardoso, L. F., de S. Ribeiro, J., Santos, V. C. A., Silva, R. L., Mota, M. P., Prudêncio, R. B., and Alves, R. C. (2022). Explanation-by-example based on item response theory. In Intelligent Systems: 11th Brazilian Conference, BRACIS 2022, Campinas, Brazil, November 28–December 1, 2022, Proceedings, Part I, pages 283–297. Springer.
Cardoso, L. F., Santos, V. C., Francês, R. S. K., Prudêncio, R. B., and Alves, R. C. (2020). Decoding machine learning benchmarks. In Brazilian Conference on Intelligent Systems, pages 412–425. Springer.
Castillo Camacho, I. and Wang, K. (2019). A simple and effective initialization of cnn for forensics of image processing operations. In Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, pages 107–112.
D’Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2022). Underspecification presents challenges for credibility in modern machine learning. The Journal of Machine Learning Research, 23(1):10237–10297.
de Andrade, D. F., Tavares, H. R., and da Cunha Valle, R. (2000). Teoria da resposta ao item: conceitos e aplicações. ABE, Sao Paulo.
He, X., Zhao, K., and Chu, X. (2021). Automl: A survey of the state-of-the-art. Knowledge-Based Systems, 212:106622.
INEP (2012). NOTA TÉCNICA: Teoria de Resposta ao Item.
Jin, H., Song, Q., and Hu, X. (2019). Auto-keras: An efficient neural architecture search system. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1946–1956.
Liu, Y., Sun, Y., Xue, B., Zhang, M., Yen, G. G., and Tan, K. C. (2021). A survey on evolutionary neural architecture search. IEEE transactions on neural networks and learning systems.
Martínez-Plumed, F., Prudêncio, R. B., Martínez-Usó, A., and Hernández-Orallo, J. (2019). Item response theory in ai: Analysing machine learning classifiers at the instance level. Artificial intelligence, 271:18–42.
Neto, H. A., Alves, R. C., and Campos, S. V. (2020). Nasirt: Automl based learning with instance-level complexity information. arXiv preprint arXiv:2008.11846.
Rasch, G. (1960). Studies in mathematical psychology: I. probabilistic models for some intelligence and attainment tests.
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Chen, X., and Wang, X. (2021). A comprehensive survey of neural architecture search: Challenges and solutions. ACM Computing Surveys (CSUR), 54(4):1–34.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.
Vanschoren, J., Van Rijn, J. N., Bischl, B., and Torgo, L. (2014). Openml: networked science in machine learning. ACM SIGKDD Explorations Newsletter, 15(2):49–60.
Cardoso, L. F., de S. Ribeiro, J., Santos, V. C. A., Silva, R. L., Mota, M. P., Prudêncio, R. B., and Alves, R. C. (2022). Explanation-by-example based on item response theory. In Intelligent Systems: 11th Brazilian Conference, BRACIS 2022, Campinas, Brazil, November 28–December 1, 2022, Proceedings, Part I, pages 283–297. Springer.
Cardoso, L. F., Santos, V. C., Francês, R. S. K., Prudêncio, R. B., and Alves, R. C. (2020). Decoding machine learning benchmarks. In Brazilian Conference on Intelligent Systems, pages 412–425. Springer.
Castillo Camacho, I. and Wang, K. (2019). A simple and effective initialization of cnn for forensics of image processing operations. In Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, pages 107–112.
D’Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2022). Underspecification presents challenges for credibility in modern machine learning. The Journal of Machine Learning Research, 23(1):10237–10297.
de Andrade, D. F., Tavares, H. R., and da Cunha Valle, R. (2000). Teoria da resposta ao item: conceitos e aplicações. ABE, Sao Paulo.
He, X., Zhao, K., and Chu, X. (2021). Automl: A survey of the state-of-the-art. Knowledge-Based Systems, 212:106622.
INEP (2012). NOTA TÉCNICA: Teoria de Resposta ao Item.
Jin, H., Song, Q., and Hu, X. (2019). Auto-keras: An efficient neural architecture search system. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1946–1956.
Liu, Y., Sun, Y., Xue, B., Zhang, M., Yen, G. G., and Tan, K. C. (2021). A survey on evolutionary neural architecture search. IEEE transactions on neural networks and learning systems.
Martínez-Plumed, F., Prudêncio, R. B., Martínez-Usó, A., and Hernández-Orallo, J. (2019). Item response theory in ai: Analysing machine learning classifiers at the instance level. Artificial intelligence, 271:18–42.
Neto, H. A., Alves, R. C., and Campos, S. V. (2020). Nasirt: Automl based learning with instance-level complexity information. arXiv preprint arXiv:2008.11846.
Rasch, G. (1960). Studies in mathematical psychology: I. probabilistic models for some intelligence and attainment tests.
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Chen, X., and Wang, X. (2021). A comprehensive survey of neural architecture search: Challenges and solutions. ACM Computing Surveys (CSUR), 54(4):1–34.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.
Vanschoren, J., Van Rijn, J. N., Bischl, B., and Torgo, L. (2014). Openml: networked science in machine learning. ACM SIGKDD Explorations Newsletter, 15(2):49–60.
Publicado
25/09/2023
Como Citar
CARDOSO, Lucas F. F.; SANTOS, Vitor C. A.; KAWASAKI FRANCÊS, Regiane S.; ALVES, Ronnie C. O..
AutoML Optimized by Genetic Algorithm and Item Response Theory. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 20. , 2023, Belo Horizonte/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 1089-1098.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2023.234616.
