Evaluating the Impact of Textual Attributes on Distance-Based Learning from Heterogeneous Data Using Text Embeddings and PCA
Resumo
Conjuntos de dados frequentemente combinam atributos numéricos, categóricos e textuais, porém abordagens para dados mistos concentram-se principalmente na combinação de atributos numéricos e categóricos. Este trabalho investiga o impacto da incorporação de atributos textuais em bases de dados estruturadas para tarefas de aprendizado de máquina baseadas em distância. Para isso, propõe-se um pipeline híbrido que representa textos por embeddings e PCA e os integra aos demais atributos. A abordagem foi avaliada em três bases de dados, utilizando algoritmos de agrupamento e classificação. Os resultados mostram que o impacto dos atributos textuais varia conforme a base e o algoritmo. Na classificação, sua inclusão apresentou ganhos significativos em cinco das seis comparações com atributos numéricos e categóricos, destacando-se a base E-Commerce, na qual o F1-Macro do KNN aumentou de 0, 349 para 0, 516 (47, 9%).Referências
Agresti, A. (2013). Categorical data analysis. John Wiley & Sons.
Ahmad, A. and Dey, L. (2007). A k-mean clustering algorithm for mixed numeric and categorical data. Data amp; Knowledge Engineering, 63(2):503–527.
Amir Ahmad, L. D. (2007). A method to compute distance between two categorical values of same attribute in unsupervised learning for categorical data set. Pattern Recognition Letters, 28(1):110–118.
Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3):273–297.
Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1):21–27.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186.
Dias, H. and Ulbrich, A. H. D. P. d. (2022). BRATECA (Brazilian Tertiary Care Dataset): a Clinical Information Dataset for the Portuguese Language. PhysioNet. Version 1.0.
Dinh, D.-T., Huynh, V.-N., and Sriboonchitta, S. (2021). Clustering mixed numerical and categorical data with missing values. Information Sciences, 571:418–442.
Fábio Souza, Rodrigo Nogueira, R. L. (2020). Bertimbau: Pretrained bert models for brazilian portuguese. Lecture Notes in Computer Science, page 403–417.
Grandini, M., Bagli, E., and Visani, G. (2020). Metrics for multi-class classification: an overview.
Guo, C., Tang, H., Niu, B., and Zhong, H. (2025). Clustering mixed data using multi-objective bacterial colony optimization with diverse learning mechanisms. Swarm and Evolutionary Computation, 98:102119.
Huang, Z. (1998). Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Mining and Knowledge Discovery, 2(3):283–304.
Isler, C. (2025). Words matter: Forecasting economic downside risks with corporate textual data. arXiv preprint arXiv:2511.04935.
K. Balaji, K. L. (2018). Clustering algorithms for mixed datasets: a review. Int J Pure Appl Math, 18(7):547–56.
Kaufman, L. and Rousseeuw, P. J. (2009). Finding groups in data: an introduction to cluster analysis. John Wiley & Sons.
Krones, F., Marikkar, U., Parsons, G., Szmul, A., and Mahdi, A. (2025). Review of multimodal machine learning approaches in healthcare. Information Fusion, 114:102690.
Lahat, D., Adali, T., and Jutten, C. (2015). Multimodal data fusion: an overview of methods, challenges, and prospects. Proceedings of the IEEE, 103(9):1449–1477.
Mehta, V., Bawa, S., and Singh, J. (2021). Weclustering: word embeddings based text clustering technique for large datasets. Complex amp; Intelligent Systems, 7(6):3211–3224.
Norris, S. and Maier, C. D. (2014). Interactions, images and texts: A reader in multimodality, volume 11. Walter de Gruyter GmbH & Co KG.
Parsons, L., Haque, E., and Liu, H. (2004). Subspace clustering for high dimensional data: A review. SIGKDD Explorations, 6(1):90–105.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830.
Rajvardhan Patil, Sorio Boit, V. G. J. N. (2023). A survey of text representation and embedding techniques in nlp. IEEE Access, 11:36120–36146.
Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 3982–3992.
Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20:53–65.
Van de Velden, M., Iodice D’Enza, A., and Markos, A. (2019). Distance-based clustering of mixed data. Wiley Interdisciplinary Reviews: Computational Statistics, 11(3):e1456.
Wang, H. and Mi, J. (2025). Intuitive-k-prototypes: A mixed data clustering algorithm with intuitionistic distribution centroid. Pattern Recognition, 158:111062.
Witten, I. H. and Frank, E. (2002). Data mining: practical machine learning tools and techniques with java implementations. Acm Sigmod Record, 31(1):76–77.
Zhang, S., Li, X., Zong, M., Zhu, X., and Cheng, D. (2017). Learning k for knn classification. ACM Transactions on Intelligent Systems and Technology (TIST), 8(3):1–19.
Ahmad, A. and Dey, L. (2007). A k-mean clustering algorithm for mixed numeric and categorical data. Data amp; Knowledge Engineering, 63(2):503–527.
Amir Ahmad, L. D. (2007). A method to compute distance between two categorical values of same attribute in unsupervised learning for categorical data set. Pattern Recognition Letters, 28(1):110–118.
Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3):273–297.
Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1):21–27.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186.
Dias, H. and Ulbrich, A. H. D. P. d. (2022). BRATECA (Brazilian Tertiary Care Dataset): a Clinical Information Dataset for the Portuguese Language. PhysioNet. Version 1.0.
Dinh, D.-T., Huynh, V.-N., and Sriboonchitta, S. (2021). Clustering mixed numerical and categorical data with missing values. Information Sciences, 571:418–442.
Fábio Souza, Rodrigo Nogueira, R. L. (2020). Bertimbau: Pretrained bert models for brazilian portuguese. Lecture Notes in Computer Science, page 403–417.
Grandini, M., Bagli, E., and Visani, G. (2020). Metrics for multi-class classification: an overview.
Guo, C., Tang, H., Niu, B., and Zhong, H. (2025). Clustering mixed data using multi-objective bacterial colony optimization with diverse learning mechanisms. Swarm and Evolutionary Computation, 98:102119.
Huang, Z. (1998). Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Mining and Knowledge Discovery, 2(3):283–304.
Isler, C. (2025). Words matter: Forecasting economic downside risks with corporate textual data. arXiv preprint arXiv:2511.04935.
K. Balaji, K. L. (2018). Clustering algorithms for mixed datasets: a review. Int J Pure Appl Math, 18(7):547–56.
Kaufman, L. and Rousseeuw, P. J. (2009). Finding groups in data: an introduction to cluster analysis. John Wiley & Sons.
Krones, F., Marikkar, U., Parsons, G., Szmul, A., and Mahdi, A. (2025). Review of multimodal machine learning approaches in healthcare. Information Fusion, 114:102690.
Lahat, D., Adali, T., and Jutten, C. (2015). Multimodal data fusion: an overview of methods, challenges, and prospects. Proceedings of the IEEE, 103(9):1449–1477.
Mehta, V., Bawa, S., and Singh, J. (2021). Weclustering: word embeddings based text clustering technique for large datasets. Complex amp; Intelligent Systems, 7(6):3211–3224.
Norris, S. and Maier, C. D. (2014). Interactions, images and texts: A reader in multimodality, volume 11. Walter de Gruyter GmbH & Co KG.
Parsons, L., Haque, E., and Liu, H. (2004). Subspace clustering for high dimensional data: A review. SIGKDD Explorations, 6(1):90–105.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830.
Rajvardhan Patil, Sorio Boit, V. G. J. N. (2023). A survey of text representation and embedding techniques in nlp. IEEE Access, 11:36120–36146.
Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 3982–3992.
Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20:53–65.
Van de Velden, M., Iodice D’Enza, A., and Markos, A. (2019). Distance-based clustering of mixed data. Wiley Interdisciplinary Reviews: Computational Statistics, 11(3):e1456.
Wang, H. and Mi, J. (2025). Intuitive-k-prototypes: A mixed data clustering algorithm with intuitionistic distribution centroid. Pattern Recognition, 158:111062.
Witten, I. H. and Frank, E. (2002). Data mining: practical machine learning tools and techniques with java implementations. Acm Sigmod Record, 31(1):76–77.
Zhang, S., Li, X., Zong, M., Zhu, X., and Cheng, D. (2017). Learning k for knn classification. ACM Transactions on Intelligent Systems and Technology (TIST), 8(3):1–19.
Publicado
19/10/2026
Como Citar
SALES, Bruno O.; SILVA, Mirlei M.; PITA, Robespierre.
Evaluating the Impact of Textual Attributes on Distance-Based Learning from Heterogeneous Data Using Text Embeddings and PCA. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 3415-3426.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.31397.
