Metodologia para Geração de Datasets de Segmentação Multimodal a partir de Atributos Geométricos e Prompts Sintéticos
Resumo
O artigo apresenta uma metodologia para mitigar a escassez de datasets multimodais, essenciais para o avanço dos Vision-Language Segmentation Models. A proposta centraliza-se na conversão de datasets de visão computacional já existentes para um formato multimodal. Através de um processo automatizado gera descrições textuais a partir das anotações visuais. Atributos cruciais do objeto, incluindo seu tamanho, localização na imagem, são combinados para a criação de prompts. A capacidade de produzir sistematicamente esses dados multimodais em larga escala a partir de recursos já anotados contribui para acelerar significativamente a pesquisa de modelos de segmentação semântica que compreendem a interação entre visão e linguagem.Referências
Amin, J., Sharif, M., Haldorai, A., Yasmin, M., and Nayak, R. S. (2022). Brain tumor detection and classification using machine learning: a comprehensive survey. Complex & intelligent systems, 8(4):3161–3183.
de Oliveira Santos, M., de Lima, F. C. d. S., Martins, L. F. L., Oliveira, J. F. P., de Almeida, L. M., and de Camargo Cancela, M. (2023). Estimativa de incidência de câncer no brasil, 2023-2025. Revista Brasileira de Cancerologia, 69(1).
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations.
Hao, S., Zhou, Y., and Guo, Y. (2020). A brief survey on semantic segmentation with deep learning. Neurocomputing, 406:302–321.
Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., and Pedrycz, W. (2024). A comprehensive survey on applications of transformers for deep learning tasks. Expert Systems with Applications, 241:122666.
Jha, D., Smedsrud, P. H., Riegler, M. A., Halvorsen, P., de Lange, T., Johansen, D., and Johansen, H. D. (2020). Kvasir-seg: A segmented polyp dataset. In MultiMedia Modeling: 26th International Conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, Proceedings, Part II, page 451–462, Berlin, Heidelberg. Springer-Verlag.
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436–444.
Li, L. and Iskander, M. (2022). Use of machine learning for classification of sand particles. Acta Geotechnica, 17(10):4739–4759.
Li, Z., Li, Y., Li, Q., Wang, P., Guo, D., Lu, L., Jin, D., Zhang, Y., and Hong, Q. (2023). Lvit: language meets vision transformer in medical image segmentation. IEEE transactions on medical imaging, 43(1):96–107.
Lloyd, S. (1982). Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129–137.
Lüddecke, T. and Ecker, A. (2022). Image segmentation using text and image prompts. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7076–7086.
Poudel, K., Dhakal, M., Bhandari, P., Adhikari, R., Thapaliya, S., and Khanal, B. (2024). Exploring transfer learning in medical image segmentation using vision-language models. In Burgos, N., Petitjean, C., Vakalopoulou, M., Christodoulidis, S., Coupe, P., Delingette, H., Lartizien, C., and Mateus, D., editors, Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, volume 250 of Proceedings of Machine Learning Research, pages 1142–1165. PMLR.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning.
Rao, Y., Zhao, W., Chen, G., Tang, Y., Zhu, Z., Huang, G., Zhou, J., and Lu, J. (2022). Denseclip: Language-guided dense prediction with context-aware prompting. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18061–18070.
Reynolds, D. (2015). Gaussian mixture models. In Encyclopedia of biometrics, pages 827–832. Springer.
Siegel, R., Kratzer, T., Giaquinto, A., Sung, H., and Jemal, A. (2025). Cancer statistics, 2025. CA: A Cancer Journal for Clinicians, 75(1):10–45.
Sokal, R. R., Michener, C. D., et al. (1958). A statistical method for evaluating systematic relationships.
Wang, Y., Su, J., Li, X., and Nakahara, E. (2025). Medlangvit: A language–vision network for medical image segmentation. Electronics, 14(15):3020.
Wang, Z., Lu, Y., Li, Q., Tao, X., Guo, Y., Gong, M., and Liu, T. (2022). Cris: Clip-driven referring image segmentation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11676–11685.
de Oliveira Santos, M., de Lima, F. C. d. S., Martins, L. F. L., Oliveira, J. F. P., de Almeida, L. M., and de Camargo Cancela, M. (2023). Estimativa de incidência de câncer no brasil, 2023-2025. Revista Brasileira de Cancerologia, 69(1).
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations.
Hao, S., Zhou, Y., and Guo, Y. (2020). A brief survey on semantic segmentation with deep learning. Neurocomputing, 406:302–321.
Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., and Pedrycz, W. (2024). A comprehensive survey on applications of transformers for deep learning tasks. Expert Systems with Applications, 241:122666.
Jha, D., Smedsrud, P. H., Riegler, M. A., Halvorsen, P., de Lange, T., Johansen, D., and Johansen, H. D. (2020). Kvasir-seg: A segmented polyp dataset. In MultiMedia Modeling: 26th International Conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, Proceedings, Part II, page 451–462, Berlin, Heidelberg. Springer-Verlag.
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436–444.
Li, L. and Iskander, M. (2022). Use of machine learning for classification of sand particles. Acta Geotechnica, 17(10):4739–4759.
Li, Z., Li, Y., Li, Q., Wang, P., Guo, D., Lu, L., Jin, D., Zhang, Y., and Hong, Q. (2023). Lvit: language meets vision transformer in medical image segmentation. IEEE transactions on medical imaging, 43(1):96–107.
Lloyd, S. (1982). Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129–137.
Lüddecke, T. and Ecker, A. (2022). Image segmentation using text and image prompts. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7076–7086.
Poudel, K., Dhakal, M., Bhandari, P., Adhikari, R., Thapaliya, S., and Khanal, B. (2024). Exploring transfer learning in medical image segmentation using vision-language models. In Burgos, N., Petitjean, C., Vakalopoulou, M., Christodoulidis, S., Coupe, P., Delingette, H., Lartizien, C., and Mateus, D., editors, Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, volume 250 of Proceedings of Machine Learning Research, pages 1142–1165. PMLR.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning.
Rao, Y., Zhao, W., Chen, G., Tang, Y., Zhu, Z., Huang, G., Zhou, J., and Lu, J. (2022). Denseclip: Language-guided dense prediction with context-aware prompting. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18061–18070.
Reynolds, D. (2015). Gaussian mixture models. In Encyclopedia of biometrics, pages 827–832. Springer.
Siegel, R., Kratzer, T., Giaquinto, A., Sung, H., and Jemal, A. (2025). Cancer statistics, 2025. CA: A Cancer Journal for Clinicians, 75(1):10–45.
Sokal, R. R., Michener, C. D., et al. (1958). A statistical method for evaluating systematic relationships.
Wang, Y., Su, J., Li, X., and Nakahara, E. (2025). Medlangvit: A language–vision network for medical image segmentation. Electronics, 14(15):3020.
Wang, Z., Lu, Y., Li, Q., Tao, X., Guo, Y., Gong, M., and Liu, T. (2022). Cris: Clip-driven referring image segmentation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11676–11685.
Publicado
01/06/2026
Como Citar
NAVES, Alexandre Arantes; FRANCO, Ricardo Augusto Pereira.
Metodologia para Geração de Datasets de Segmentação Multimodal a partir de Atributos Geométricos e Prompts Sintéticos. In: SIMPÓSIO BRASILEIRO DE COMPUTAÇÃO APLICADA À SAÚDE (SBCAS), 26. , 2026, Ouro Preto/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 1038-1049.
ISSN 2763-8952.
DOI: https://doi.org/10.5753/sbcas.2026.21617.
