GA-Channel-Xception: Genetic Algorithm-Based Color Channel Selection for Deep Learning Detection of Oral Squamous Cell Carcinoma
Resumo
Oral Squamous Cell Carcinoma (OSCC) is a prevalent malignancy with high mortality due to late-stage diagnosis, driven by challenges in interpreting subtle histopathological image features. We introduce GA-Channel-Xception, a novel bioinformatics framework that integrates a Genetic Algorithm (GA) with the Xception convolutional neural network to enhance the detection of OSCC from histopathological images. Our approach leverages GA to optimize color channel selection across RGB, HSV, and LUV color spaces, identifying the most informative three-channel combination for classification. A tailored Xception model, trained on synthetic images derived from these channels, was evaluated on a publicly available OSCC histopathological dataset. The proposed method achieved an accuracy of 98.71% and an F1-score of 96.77%, outperforming conventional approaches. These results underscore the potential of integrating evolutionary algorithms with deep learning to improve automated histopathological analysis, offering a scalable tool for early OSCC diagnosis and precision oncology.
Palavras-chave:
Deep Learning, Genetic Algorithm, Oral Cancer
Referências
Alvino, A. F., Alves, E. S., Brito, S. S., Nascimento, V. T., da Cruz, L. B., Diniz, J. O., Souza Jr, L. O., da Silva, J. C., and Gomes Jr, D. L. (2025). Abordagem baseada em deep features para diagnóstico de câncer seroso de ovário em imagens histopatológicas. In Simpósio Brasileiro de Computação Aplicada à Saúde (SBCAS), pages 401–412. SBC.
Bisht, S. R., Mishra, P., Yadav, D., Rawal, R., and Mercado-Shekhar, K. P. (2021). Current and emerging techniques for oral cancer screening and diagnosis: a review. Progress in Biomedical Engineering, 3(4):042003.
Carvalho, E. D., Antonio Filho, O., Silva, R. R., Araujo, F. H., Diniz, J. O., Silva, A. C., Paiva, A. C., and Gattass, M. (2020). Breast cancer diagnosis from histopathological images using textural features and cbir. Artificial intelligence in medicine, 105:101845.
Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1251–1258.
Das, M., Dash, R., Mishra, S. K., and Dalai, A. K. (2024). An ensemble deep learning model for oral squamous cell carcinoma detection using histopathological image analysis. IEEE Access.
Deif, M. A., Attar, H., Amer, A., Elhaty, I. A., Khosravi, M. R., and Solyman, A. A. (2022). Diagnosis of oral squamous cell carcinoma using deep neural networks and binary particle swarm optimization on histopathological images: an aiomt approach. Computational Intelligence and Neuroscience, 2022(1):6364102.
Diniz, J. O., Ribeiro, N. P., Junior, D. A. D., da Cruz, L. B., de Carvalho Filho, A. O., Gomes Jr, D. L., Silva, A. C., and de Paiva, A. C. (2024a). Efficientxyz-deepfeatures: seleção de esquema de cor e arquitetura deep features na classificação de câncer de cólon em imagens histopatológicas. In Simpósio Brasileiro de Computação Aplicada à Saúde (SBCAS), pages 82–93. SBC.
Diniz, J. O. B., Ribeiro, N. P., Dias Jr, D. A., da Cruz, L. B., da Silva, G. L., Gomes Jr, D. L., de Paiva, A. C., and Silva, A. C. (2024b). Anisotropicbreast-vit: Breast cancer classification in ultrasound images using anisotropic filtering and vision transformer. In Brazilian Conference on Intelligent Systems, pages 95–109. Springer.
Eckert, A. W., Kappler, M., Große, I., Wickenhauser, C., and Seliger, B. (2020). Current understanding of the hif-1-dependent metabolism in oral squamous cell carcinoma. International journal of molecular sciences, 21(17):6083.
Gonzalez, R. and Woods, R. (2008). Digital image processing. Pearson, Prentice Hall.
INCA (2023). Instituto nacional de câncer, estimativa 2023: Incidência de câncer no brasil. [link]. Accessed on: June 26. 2025.
Júnior, D. A. D., da Cruz, L. B., Diniz, J. O. B., da Silva, G. L. F., Junior, G. B., Silva, A. C., de Paiva, A. C., Nunes, R. A., and Gattass, M. (2021). Automatic method for classifying covid-19 patients based on chest x-ray images, using deep features and pso-optimized xgboost. Expert Systems with Applications, 183:115452.
Kebede, A. F. (2022). Histopathologic oral cancer detection using cnns. [link]. Accessed on: June 26. 2025.
Kumar, A. and Nelson, L. (2025). Enhancing oral squamous cell carcinoma detection using efficientnetb3 from histopathologic images. In 2025 International Conference on Multi-Agent Systems for Collaborative Intelligence (ICMSCI), pages 950–956. IEEE.
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, J. A., Van Ginneken, B., and Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88.
Maia, B. M. S., de Assis, M. C. F. R., de Lima, L. M., Rocha, M. B., Calente, H. G., Correa, M. L. A., Camisasca, D. R., and Krohling, R. A. (2024). Transformers, convolutional neural networks, and few-shot learning for classification of histopathological images of oral cancer. Expert Systems with Applications, 241:122418.
Murthy, A. S. R. C., Mercy, G., Prakash, L. J., and Bose, K. S. (2025). Histopath-dl-oc: Deep learning for oral cancer prediction from histopathology data. In 2025 International Conference on Inventive Computation Technologies (ICICT), pages 1190–1197. IEEE.
Powers, D. M. (2020). Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061.
Prado, R. L. d., Marsicano, J. A., Frois, A. K., and Brancher, J. D. (2025). The use of machine learning to support the diagnosis of oral alterations. Pesquisa Brasileira em Odontopediatria e Clínica Integrada, 25:e240048.
Rahman, A.-u., Alqahtani, A., Aldhafferi, N., Nasir, M. U., Khan, M. F., Khan, M. A., and Mosavi, A. (2022). Histopathologic oral cancer prediction using oral squamous cell carcinoma biopsy empowered with transfer learning. Sensors, 22(10):3833.
Raval, D., Patel, A., Undavia, J. N., Shukla, A., and Patel, U. (2024). Oral cancer detection with convolutional neural networks and transfer learning: A resnet-based approach. In International Conference on Data Analytics & Management, pages 237–245. Springer.
Ribeiro, N. P., Teles, F. R., Diniz, J. O. B., da Cruz, L. B., Dias Jr, D. A., Braz Junior, G., de Almeida, J. D., and de Paiva, A. C. (2024). Improving colorectal cancer diagnosis using mirnet and inceptionv3 on histopathological images. In Brazilian Conference on Intelligent Systems, pages 321–334. Springer.
Shapiro, J. (1999). Genetic algorithms in machine learning. In Advanced course on artificial intelligence, pages 146–168. Springer.
Tamanini, B. A., Sousa, V. G., Rodrigues, L. P., Oliveira, D. M., Dias, C. X., da Cruz, L. B., Diniz, J. O., and Júnior, L. O. S. (2025). Classificaçao de carcinoma endometrioide de ovário por transformaçao de esquema de cor e radiomics em imagens histopatológicas. In Simpósio Brasileiro de Computação Aplicada à Saúde (SBCAS), pages 68–79. SBC.
WHO (2024). World health organization: Oral cancer. [link]. Accessed on: June 26. 2025.
Bisht, S. R., Mishra, P., Yadav, D., Rawal, R., and Mercado-Shekhar, K. P. (2021). Current and emerging techniques for oral cancer screening and diagnosis: a review. Progress in Biomedical Engineering, 3(4):042003.
Carvalho, E. D., Antonio Filho, O., Silva, R. R., Araujo, F. H., Diniz, J. O., Silva, A. C., Paiva, A. C., and Gattass, M. (2020). Breast cancer diagnosis from histopathological images using textural features and cbir. Artificial intelligence in medicine, 105:101845.
Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1251–1258.
Das, M., Dash, R., Mishra, S. K., and Dalai, A. K. (2024). An ensemble deep learning model for oral squamous cell carcinoma detection using histopathological image analysis. IEEE Access.
Deif, M. A., Attar, H., Amer, A., Elhaty, I. A., Khosravi, M. R., and Solyman, A. A. (2022). Diagnosis of oral squamous cell carcinoma using deep neural networks and binary particle swarm optimization on histopathological images: an aiomt approach. Computational Intelligence and Neuroscience, 2022(1):6364102.
Diniz, J. O., Ribeiro, N. P., Junior, D. A. D., da Cruz, L. B., de Carvalho Filho, A. O., Gomes Jr, D. L., Silva, A. C., and de Paiva, A. C. (2024a). Efficientxyz-deepfeatures: seleção de esquema de cor e arquitetura deep features na classificação de câncer de cólon em imagens histopatológicas. In Simpósio Brasileiro de Computação Aplicada à Saúde (SBCAS), pages 82–93. SBC.
Diniz, J. O. B., Ribeiro, N. P., Dias Jr, D. A., da Cruz, L. B., da Silva, G. L., Gomes Jr, D. L., de Paiva, A. C., and Silva, A. C. (2024b). Anisotropicbreast-vit: Breast cancer classification in ultrasound images using anisotropic filtering and vision transformer. In Brazilian Conference on Intelligent Systems, pages 95–109. Springer.
Eckert, A. W., Kappler, M., Große, I., Wickenhauser, C., and Seliger, B. (2020). Current understanding of the hif-1-dependent metabolism in oral squamous cell carcinoma. International journal of molecular sciences, 21(17):6083.
Gonzalez, R. and Woods, R. (2008). Digital image processing. Pearson, Prentice Hall.
INCA (2023). Instituto nacional de câncer, estimativa 2023: Incidência de câncer no brasil. [link]. Accessed on: June 26. 2025.
Júnior, D. A. D., da Cruz, L. B., Diniz, J. O. B., da Silva, G. L. F., Junior, G. B., Silva, A. C., de Paiva, A. C., Nunes, R. A., and Gattass, M. (2021). Automatic method for classifying covid-19 patients based on chest x-ray images, using deep features and pso-optimized xgboost. Expert Systems with Applications, 183:115452.
Kebede, A. F. (2022). Histopathologic oral cancer detection using cnns. [link]. Accessed on: June 26. 2025.
Kumar, A. and Nelson, L. (2025). Enhancing oral squamous cell carcinoma detection using efficientnetb3 from histopathologic images. In 2025 International Conference on Multi-Agent Systems for Collaborative Intelligence (ICMSCI), pages 950–956. IEEE.
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, J. A., Van Ginneken, B., and Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88.
Maia, B. M. S., de Assis, M. C. F. R., de Lima, L. M., Rocha, M. B., Calente, H. G., Correa, M. L. A., Camisasca, D. R., and Krohling, R. A. (2024). Transformers, convolutional neural networks, and few-shot learning for classification of histopathological images of oral cancer. Expert Systems with Applications, 241:122418.
Murthy, A. S. R. C., Mercy, G., Prakash, L. J., and Bose, K. S. (2025). Histopath-dl-oc: Deep learning for oral cancer prediction from histopathology data. In 2025 International Conference on Inventive Computation Technologies (ICICT), pages 1190–1197. IEEE.
Powers, D. M. (2020). Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061.
Prado, R. L. d., Marsicano, J. A., Frois, A. K., and Brancher, J. D. (2025). The use of machine learning to support the diagnosis of oral alterations. Pesquisa Brasileira em Odontopediatria e Clínica Integrada, 25:e240048.
Rahman, A.-u., Alqahtani, A., Aldhafferi, N., Nasir, M. U., Khan, M. F., Khan, M. A., and Mosavi, A. (2022). Histopathologic oral cancer prediction using oral squamous cell carcinoma biopsy empowered with transfer learning. Sensors, 22(10):3833.
Raval, D., Patel, A., Undavia, J. N., Shukla, A., and Patel, U. (2024). Oral cancer detection with convolutional neural networks and transfer learning: A resnet-based approach. In International Conference on Data Analytics & Management, pages 237–245. Springer.
Ribeiro, N. P., Teles, F. R., Diniz, J. O. B., da Cruz, L. B., Dias Jr, D. A., Braz Junior, G., de Almeida, J. D., and de Paiva, A. C. (2024). Improving colorectal cancer diagnosis using mirnet and inceptionv3 on histopathological images. In Brazilian Conference on Intelligent Systems, pages 321–334. Springer.
Shapiro, J. (1999). Genetic algorithms in machine learning. In Advanced course on artificial intelligence, pages 146–168. Springer.
Tamanini, B. A., Sousa, V. G., Rodrigues, L. P., Oliveira, D. M., Dias, C. X., da Cruz, L. B., Diniz, J. O., and Júnior, L. O. S. (2025). Classificaçao de carcinoma endometrioide de ovário por transformaçao de esquema de cor e radiomics em imagens histopatológicas. In Simpósio Brasileiro de Computação Aplicada à Saúde (SBCAS), pages 68–79. SBC.
WHO (2024). World health organization: Oral cancer. [link]. Accessed on: June 26. 2025.
Publicado
29/09/2025
Como Citar
DINIZ, João O. B.; TAMANINI, Breno A.; ALVINO, André F.; CRUZ, Luana B. da; DA SILVA, Josenildo C.; CORTES, Omar A. C.; GOMES JR, Daniel L.; DE CARVALHO FILHO, Antônio O..
GA-Channel-Xception: Genetic Algorithm-Based Color Channel Selection for Deep Learning Detection of Oral Squamous Cell Carcinoma. In: SIMPÓSIO BRASILEIRO DE BIOINFORMÁTICA (BSB), 18. , 2025, Fortaleza/CE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 106-117.
ISSN 2316-1248.
DOI: https://doi.org/10.5753/bsb.2025.14911.
