Performance Variability of Machine Learning Models using Limited Data for Collusion Detection: A Case Study of the Brazilian Car Wash Operation

• Everton Schneider dos Santos Universidade Federal de Santa Catarina (UFSC)
• Matheus Machado dos Santos Universidade Federal de Santa Catarina (UFSC)
• Márcio Castro Universidade Federal de Santa Catarina (UFSC)
• Jonata Tyska Carvalho Universidade Federal de Santa Catarina (UFSC)

DOI: https://doi.org/10.5753/sbbd.2024.240845

Resumo

Fraudulent companies form illegal agreements, like collusion and cartels, to circumvent the impartiality and competitiveness of the public procurement auctions. These types of fraud can cause significant financial losses and erode trust in the public sector. Therefore, building reliable methods for early detection of frauds is a priority for public organizations. This study uses an enriched version of the “Operation Car Wash” dataset to evaluate the collusion detection capabilities of different machine learning algorithms. Using cross-validation techniques, the methodology proposed in our work was able to improve the collusion detection rate of the learning models used in this work, outperforming the results of other works found in the literature.

Palavras-chave: public procurement, fraud detection, model selection

Referências

Aggarwal, C. C. et al. (2015). Data mining: the textbook, volume 1. Springer.

Curtis, F. and Maines, P. (1973). Closed competitive bidding. Omega, 1(5):613–619.

Decarolis, F. and Giorgiantonio, C. (2022). Corruption red flags in public procurement: new evidence from italian calls for tenders. EPJ Data Science, 11(1):16.

García Rodríguez, M. J., Rodríguez-Montequín, V., Ballesteros-Pérez, P., Love, P. E., and Signor, R. (2022). Collusion detection in public procurement auctions with machine learning algorithms. Automation in Construction, 133:104047.

Grandini, M., Bagli, E., and Visani, G. (2020). Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756.

Huber, M. and Imhof, D. (2019). Machine learning with screens for detecting bid-rigging cartels. International Journal of Industrial Organization, 65:277–301.

Huber, M., Imhof, D., and Ishii, R. (2022). Transnational machine learning with screens for flagging bid-rigging cartels. Journal of the Royal Statistical Society Series A: Statistics in Society, 185(3):1074–1114.

Imhof, D. and Wallimann, H. (2021). Detecting bid-rigging coalitions in different countries and auction formats. International Review of Law and Economics, 68:106016.

Korjus, K., Hebart, M. N., and Vicente, R. (2016). An efficient data partitioning to improve classification performance while keeping parameters interpretable. PloS one, 11(8):e0161788.

Lyra, M. S., Curado, A., Damásio, B., Bação, F., and Pinheiro, F. L. (2021). Characterization of the firm–firm public procurement co-bidding network from the state of ceará (brazil) municipalities. Applied Network Science, 6:1–10.

McKinney, W. (2010). Data structures for statistical computing in python. In van der Walt, S. and Millman, J., editors, Proceedings of the 9th Python in Science Conference, Proceedings of the Python in Science Conference, pages 56–61. SciPy.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.

Porter, R. H. and Zona, J. D. (1993). Detection of bid rigging in procurement auctions. Journal of political economy, 101(3):518–538.

Signor, R., Ballesteros-Pérez, P., and Love, P. E. (2021). Collusion detection in infrastructure procurement: A modified order statistic method for uncapped auctions. IEEE transactions on engineering management, 70(2):464–477.

Signor, R., Love, P. E., Belarmino, A. T., and Alfred Olatunji, O. (2020a). Detection of collusive tenders in infrastructure projects: Learning from operation car wash. Journal of Construction Engineering and Management, 146(1):05019015.

Signor, R., Love, P. E., and Ika, L. A. (2020b). White collar crime: Unearthing collusion in the procurement of infrastructure projects. IEEE Transactions on Engineering Management, 69(5):1932–1943.

Velasco, R. B., Carpanese, I., Interian, R., Paulo Neto, O. C., and Ribeiro, C. C. (2021). A decision support system for fraud detection in public procurement. International Transactions in Operational Research, 28(1):27–47.

Villamil, I., Kertész, J., and Fazekas, M. (2024). Collusion risk in corporate networks. Scientific Reports, 14(1):3161.

Wallimann, H., Imhof, D., and Huber, M. (2023). A machine learning approach for flagging incomplete bid-rigging cartels. Computational Economics, 62(4):1669–1720.

Wallimann, H. and Sticher, S. (2023). On suspicious tracks: machine-learning based approaches to detect cartels in railway-infrastructure procurement. Transport Policy, 143:121–131.