Challenges in MLOps Adoption by DevOps Teams - co-development project between government and academia introducing e-gov3.0

• Carla Silva Rocha Aguiar University of Brasília https://orcid.org/0000-0003-3102-5166

DOI: https://doi.org/10.5753/cbsoft_estendido.2020.14623

Abstract

The adoption of machine learning systems has accelerated in recent years due to the availability of tools, frameworks, and libraries. While the implementation of a machine learning system is facilitated, the challenges related to these models' maintenance and evolution have been little discussed. Research and surveys show that engineers still have difficulty in operationalizing and standardizing processes for continuous deployment. In this context, I report my experience coordinating the adoption of MLOps during an unprecedented partnership between the government and academia for 24 months to introduce e-gov 3.0 services. From the post-mortem analysis of the data from the developed Machine learning systems projects, a chatbot, and a recommendation service, I have drawn up lessons learned and best practices for successfully adopting MLOps from a team that is already mature in DevOps culture.

Keywords: DevOps, MLOps, open source, co-development, e-government 30

References

Amershi, S., Begel, A., Bird, C., Deline, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., and Zimmermann, T. (2019b). Software engineering for machine learning: A case study. pages 291–300.

de Lacerda, A. R. T. and Aguiar, C. S. R. (2019). Floss faq chatbot project reuse: How to allow nonexperts to develop a chatbot. In Proceedings of the 15th International Symposium on Open Collaboration, OpenSym ’19, New York, NY, USA. Association for Computing Machinery.

Leite, L., Rocha, C., Kon, F., Milojicic, D., and Meirelles, P. (2019). A survey of devops concepts and challenges. ACM Comput. Surv., 52(6).

López García, , De Lucas, J. M., Antonacci, M., Zu Castell, W., David, M., Hardt, M., Lloret Iglesias, L., Moltó, G., Plociennik, M., Tran, V., Alic, A. S., Caballer, M., Plasencia, I. C., Costantini, A., Dlugolinsky, S., Duma, D. C., Donvito, G., Gomes, J., Heredia Cacha, I., Ito, K., Kozlov, V. Y., Nguyen, G., Orviz Fernández, P., Sustr, Z., and Wolniewicz, P. (2020). A cloud-based framework for machine learning workloads and applications. IEEE Access, 8:18681–18692.

Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., and Dennison, D. (2015). Hidden technical debt in machine learning systems. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS’15, page 2503–2511, Cambridge, MA, USA. MIT Press.

Wen, M., Siqueira, R., Lago, N., Camarinha, D., Terceiro, A., Kon, F., and Meirelles, P. (2020). Leading successful government-academia collaborations using floss and agile values. Journal of Systems and Software, 164:110548