Can a simple customer review outperform a feature set for predicting churn?
Resumo
A previsão de perda de clientes (churn) tradicionalmente usa dados de perfis e transações, deixando inexploradas características textuais como comentários dos clientes. Este trabalho compara modelos de aprendizado de máquina para previsão de churn que usam dados convencionais com aqueles que usam revisões postadas pelos clientes sobre suas compras. Nossos experimentos com os modelos mais utilizados para previsão de churn na literatura revelam que, usando dados convencionais, os modelos apresentam o melhor desempenho com a segmentação RFM, alcançando até 93% de F1-Score. Esse valor cai para menos de 75% sem a segmentação RFM. Em contraste, usando embeddings BERT dos textos das avaliações um F1-Score de 96% é alcançado.
Palavras-chave:
Churn prediction, Language Model, Machine Learning
Referências
Ahn, J., Hwang, J., Kim, D., Choi, H., Kang, S.: A survey on churn analysis in various business domains. IEEE Access 8, 220816–220839 (2020). DOI: 10.1109/ACCESS.2020.3042657
Aleksandrova, Y.: Application of machine learning for churn prediction based on transactional data (rfm analysis). In: 18 International Multidisciplinary Scientific Geoconference SGEM 2018: Conference Proceedings. vol. 18, pp. 125–132 (2018)
Amiri, H., Daume III, H.: Target-dependent churn classification in microblogs. Proceedings of the AAAI Conference on Artificial Intelligence 29(1) (Feb 2015). DOI: 10.1609/aaai.v29i1.9532
Amiri, H., Daume III, H.: Short text representation for detecting churn in microblogs. Proceedings of the AAAI Conference on Artificial Intelligence 30(1) (Mar 2016). DOI: 10.1609/aaai.v30i1.10333
Beckhauser, W.J., Fileto, R.: Churn prediction in enterprises with high customer turnover. In: International Conference on Information Integration and Web Intelligence. pp. 176–191. Springer (2023)
Berry, M.J., Linoff, G.S.: Data mining techniques: for marketing, sales, and customer relationship management. John Wiley & Sons (2004)
Cheng, C.H., Chen, Y.S.: Classifying the segmentation of customer value via rfm model and rs theory. Expert systems with applications 36(3), 4176–4184 (2009)
Gallo, A.: The value of keeping the right customers. Harvard Business Review (Nov 2014), [link]
Gao, L., de Haan, E., Melero-Polo, I., Sese, F.J.: Winning your customers’ minds and hearts: Disentangling the effects of lock-in and affective customer experience on retention. Journal of the Academy of Marketing Science 51(2), 334–371 (2023)
Geiler, L., Affeldt, S., Nadif, M.: A survey on machine learning methods for churn prediction. International Journal of Data Science and Analytics 14(3), 217–242 (2022)
Gridach, M., Haddad, H., Mulki, H.: Churn identification in microblogs using convolutional neural networks with structured logical knowledge (11 2017). DOI: 10.18653/v1/W17-4403
Hartmann, J., Heitmann, M., Siebert, C., Schamp, C.: More than a feeling: Accuracy and application of sentiment analysis. International Journal of Research in Marketing 40(1), 75–87 (2023). DOI: https://doi.org/10.1016/j.ijresmar.2022.05.005
Hochstein, B., Voorhees, C.M., Pratt, A.B., Rangarajan, D., Nagel, D.M., Mehrotra, V.: Customer success management, customer health, and retention in b2b industries. International Journal of Research in Marketing 40(4), 912–932 (2023)
Ibitoye, A., Onifade, O.: Sequentially clustered social opinion for improved customer management in churn prediction (06 2022). DOI: 10.21203/rs.3.rs-1797516/v1
Jain, H., Khunteta, A., Srivastava, S.: Telecom churn prediction and used techniques, datasets and performance measures: a review. Telecommunication systems 76(4), 613–630 (2021)
Koroteev, M.V.: Bert: A review of applications in natural language processing and understanding. ArXiv abs/2103.11943 (2021), [link]
Krugmann, J.O., Hartmann, J.: Sentiment analysis in the age of generative ai. Customer Needs and Solutions 11(1), 1–19 (2024)
Mena, G., Coussement, K., De Bock, K.W., De Caigny, A., Lessmann, S.: Exploiting time-varying rfm measures for customer churn prediction with deep neural networks. Annals of Operations Research pp. 1–23 (2023)
Mitrovic, S., Singh, G., Baesens, B., Lemahieu, W., de Weerdt, J.: Scalable rfm-enriched representation learning for churn prediction. In: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA). pp. 79–88 (2017). DOI: 10.1109/DSAA.2017.42
Mustafa, N., Sook Ling, L., Abdul Razak, S.F.: Customer churn prediction for telecommunication industry: A malaysian case study [version 1; peer review: 2 approved]. F1000 research 10, 1274–1274 (2021)
Nagaraj, P., Muneeswaran, V., Dharanidharan, A., Aakash, M., Balananthanan, K., Rajkumar, C.: E-commerce customer churn prediction scheme based on customer behaviour using machine learning. In: Intl. Conf. on Computer Communication and Informatics (ICCCI). pp. 1–6. IEEE (2023)
Perišić, A., Pahor, M.: Clustering mixed-type player behavior data for churn prediction in mobile games. Central European journal of operations research 31(1), 165–190 (2023)
Perišić, A., Pahor, M.: Rfm-lir feature framework for churn prediction in the mobile games market. IEEE Transactions on Games 14(2), 126–137 (2022). DOI: 10.1109/TG.2021.3067114
Shobana, J., Gangadhar, C., Arora, R.K., Renjith, P., Bamini, J., devidas Chincholkar, Y.: E-commerce customer churn prevention using machine learning-based business intelligence strategy. Measurement: Sensors 27, 100728 (2023)
Sobreiro, P., Martinho, D.D.S., Alonso, J.G., Berrocal, J.: A slr on customer dropout prediction. IEEE access 10, 14529–14547 (2022)
Suh, Y.: Machine learning based customer churn prediction in home appliance rental business. Journal of big Data 10(1), 41 (2023)
Tran, H., Le, N., Nguyen, V.H.: Customer churn prediction in the banking sector using machine learning-based classification models. Interdisciplinary Journal of Information, Knowledge & Management 18 (2023)
Wenger, M.: Strategic Business Models in the Online Food Delivery Industry-Detailed Analysis of the- ” Order and Delivery” Business Model. Master’s thesis, Universidade NOVA de Lisboa (Portugal) (2021)
Zhong, J., Li, W.: Predicting customer churn in the telecommunication industry by analyzing phone call transcripts with convolutional neural networks. In: 3rd Intl. Conf. on Innovation in Artificial Intelligence. p. 55–59. ICIAI 2019, ACM, New York, NY, USA (2019). DOI: 10.1145/3319921.3319937
Aleksandrova, Y.: Application of machine learning for churn prediction based on transactional data (rfm analysis). In: 18 International Multidisciplinary Scientific Geoconference SGEM 2018: Conference Proceedings. vol. 18, pp. 125–132 (2018)
Amiri, H., Daume III, H.: Target-dependent churn classification in microblogs. Proceedings of the AAAI Conference on Artificial Intelligence 29(1) (Feb 2015). DOI: 10.1609/aaai.v29i1.9532
Amiri, H., Daume III, H.: Short text representation for detecting churn in microblogs. Proceedings of the AAAI Conference on Artificial Intelligence 30(1) (Mar 2016). DOI: 10.1609/aaai.v30i1.10333
Beckhauser, W.J., Fileto, R.: Churn prediction in enterprises with high customer turnover. In: International Conference on Information Integration and Web Intelligence. pp. 176–191. Springer (2023)
Berry, M.J., Linoff, G.S.: Data mining techniques: for marketing, sales, and customer relationship management. John Wiley & Sons (2004)
Cheng, C.H., Chen, Y.S.: Classifying the segmentation of customer value via rfm model and rs theory. Expert systems with applications 36(3), 4176–4184 (2009)
Gallo, A.: The value of keeping the right customers. Harvard Business Review (Nov 2014), [link]
Gao, L., de Haan, E., Melero-Polo, I., Sese, F.J.: Winning your customers’ minds and hearts: Disentangling the effects of lock-in and affective customer experience on retention. Journal of the Academy of Marketing Science 51(2), 334–371 (2023)
Geiler, L., Affeldt, S., Nadif, M.: A survey on machine learning methods for churn prediction. International Journal of Data Science and Analytics 14(3), 217–242 (2022)
Gridach, M., Haddad, H., Mulki, H.: Churn identification in microblogs using convolutional neural networks with structured logical knowledge (11 2017). DOI: 10.18653/v1/W17-4403
Hartmann, J., Heitmann, M., Siebert, C., Schamp, C.: More than a feeling: Accuracy and application of sentiment analysis. International Journal of Research in Marketing 40(1), 75–87 (2023). DOI: https://doi.org/10.1016/j.ijresmar.2022.05.005
Hochstein, B., Voorhees, C.M., Pratt, A.B., Rangarajan, D., Nagel, D.M., Mehrotra, V.: Customer success management, customer health, and retention in b2b industries. International Journal of Research in Marketing 40(4), 912–932 (2023)
Ibitoye, A., Onifade, O.: Sequentially clustered social opinion for improved customer management in churn prediction (06 2022). DOI: 10.21203/rs.3.rs-1797516/v1
Jain, H., Khunteta, A., Srivastava, S.: Telecom churn prediction and used techniques, datasets and performance measures: a review. Telecommunication systems 76(4), 613–630 (2021)
Koroteev, M.V.: Bert: A review of applications in natural language processing and understanding. ArXiv abs/2103.11943 (2021), [link]
Krugmann, J.O., Hartmann, J.: Sentiment analysis in the age of generative ai. Customer Needs and Solutions 11(1), 1–19 (2024)
Mena, G., Coussement, K., De Bock, K.W., De Caigny, A., Lessmann, S.: Exploiting time-varying rfm measures for customer churn prediction with deep neural networks. Annals of Operations Research pp. 1–23 (2023)
Mitrovic, S., Singh, G., Baesens, B., Lemahieu, W., de Weerdt, J.: Scalable rfm-enriched representation learning for churn prediction. In: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA). pp. 79–88 (2017). DOI: 10.1109/DSAA.2017.42
Mustafa, N., Sook Ling, L., Abdul Razak, S.F.: Customer churn prediction for telecommunication industry: A malaysian case study [version 1; peer review: 2 approved]. F1000 research 10, 1274–1274 (2021)
Nagaraj, P., Muneeswaran, V., Dharanidharan, A., Aakash, M., Balananthanan, K., Rajkumar, C.: E-commerce customer churn prediction scheme based on customer behaviour using machine learning. In: Intl. Conf. on Computer Communication and Informatics (ICCCI). pp. 1–6. IEEE (2023)
Perišić, A., Pahor, M.: Clustering mixed-type player behavior data for churn prediction in mobile games. Central European journal of operations research 31(1), 165–190 (2023)
Perišić, A., Pahor, M.: Rfm-lir feature framework for churn prediction in the mobile games market. IEEE Transactions on Games 14(2), 126–137 (2022). DOI: 10.1109/TG.2021.3067114
Shobana, J., Gangadhar, C., Arora, R.K., Renjith, P., Bamini, J., devidas Chincholkar, Y.: E-commerce customer churn prevention using machine learning-based business intelligence strategy. Measurement: Sensors 27, 100728 (2023)
Sobreiro, P., Martinho, D.D.S., Alonso, J.G., Berrocal, J.: A slr on customer dropout prediction. IEEE access 10, 14529–14547 (2022)
Suh, Y.: Machine learning based customer churn prediction in home appliance rental business. Journal of big Data 10(1), 41 (2023)
Tran, H., Le, N., Nguyen, V.H.: Customer churn prediction in the banking sector using machine learning-based classification models. Interdisciplinary Journal of Information, Knowledge & Management 18 (2023)
Wenger, M.: Strategic Business Models in the Online Food Delivery Industry-Detailed Analysis of the- ” Order and Delivery” Business Model. Master’s thesis, Universidade NOVA de Lisboa (Portugal) (2021)
Zhong, J., Li, W.: Predicting customer churn in the telecommunication industry by analyzing phone call transcripts with convolutional neural networks. In: 3rd Intl. Conf. on Innovation in Artificial Intelligence. p. 55–59. ICIAI 2019, ACM, New York, NY, USA (2019). DOI: 10.1145/3319921.3319937
Publicado
14/10/2024
Como Citar
BECKHAUSER, William Jones; FILETO, Renato.
Can a simple customer review outperform a feature set for predicting churn?. In: SIMPÓSIO BRASILEIRO DE BANCO DE DADOS (SBBD), 39. , 2024, Florianópolis/SC.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 117-128.
ISSN 2763-8979.
DOI: https://doi.org/10.5753/sbbd.2024.240217.
