Smart Notifications – An ML-based Framework to Boost User Engagement

  • Victor M. Magalhães Pinto iFood
  • Leonardo M. Murtha Oliveira iFood
  • Ysabelle Pinheiro De Sousa iFood
  • Stefania L. Danyi iFood
  • Dimas S. Lima iFood


The use of daily push notifications is prevalent in many online and mobile applications to enhance and maintain user engagement. Push notifications are often used in customer relationship management (CRM) campaigns to promote engagement, and frequently a customer is subjected to several on a daily basis and many at the same time. This often results in multiple notifications being scheduled to a user simultaneously. Also, in online apps, push notifications can trigger new orders during various shifts throughout the day for each user. This paper presents a complete framework of push notification modeling that takes into account the human-in-the-loop aspect of the problem, mixing up modeling with business decisions. The model structure is based on a two-tower deep learning model to rank push notifications based on their relevance to users, utilizing push metadata and user features. It also analyzes the causal impact of sending push notifications during each shift of the day. We use it to successfully optimize more than 100 million daily push notifications on the food delivery app iFood, resulting in increased orders and reduced average push notifications per user.
Palavras-chave: push notification, recommendation system, causal inference


Charu C Aggarwal 2016. Recommender Systems. Springer.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

Gautam Chauhan, Dhruva Vatsa Mishra, M Farida Begam, 2019. Customer-Aware Recommender System for Push Notifications in an e-commerce Environment. In 2019 Global Conference for Advancement in Technology (GCAT). IEEE, 1–7.

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7–10.

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191–198.

Felipe Ferreira, Daniele R Souza, Igor Moura, Matheus Barbieri, and Hélio CV Lopes. 2020. Investigating multimodal features for video recommendations at globoplay. In Proceedings of the 14th ACM Conference on Recommender Systems. 571–572.

Ronald Aylmer Fisher. 1936. Design of experiments. British Medical Journal 1, 3923 (1936), 554

Diana Gavilan and Gema Martinez-Navarro. 2022. Exploring user’s experience of push notifications: a grounded theory approach. Qualitative Market Research: An International Journal (2022).

Merve Gençer, Gökhan Bilgin, Özgür Zan, and Tansel Voyvodaoğlu. 2013. A new framework for increasing user engagement in mobile applications using machine learning techniques. In Design, User Experience, and Usability. Web, Mobile, and Product Design: Second International Conference, DUXU 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part IV 2. Springer, 651–659.

Pierre Gutierrez and Jean-Yves Gérardy. 2017. Causal inference and uplift modelling: A review of the literature. In International conference on predictive applications and APIs. PMLR, 1–13

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173–182.

Fredrik Johansson, Uri Shalit, and David Sontag. 2016. Learning representations for counterfactual inference. In International conference on machine learning. PMLR, 3020–3029

Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing (2nd Edition). Prentice-Hall, Inc., USA

Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. 2017. Causal effect inference with deep latent-variable models. Advances in neural information processing systems 30 (2017)

Adithya Madhusoodanan, Anand Kumar, Kieran Fraser, and Bilal Yousuf. 2020. Machine Learning Approach to Manage Adaptive Push Notifications for Improving User Experience. In MobiQuitous 2020-17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 488–493.

Conor O’Brien, Huasen Wu, Shaodan Zhai, Dalin Guo, Wenzhe Shi, and Jonathan J Hunt. 2022. Should I send this notification? Optimizing push notifications decision making by modeling the future. arXiv preprint arXiv:2202.08812 (2022).

Tadashi Okoshi, Kota Tsubouchi, and Hideyuki Tokuda. 2019. Real-world product deployment of adaptive push notification scheduling on smartphones. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2792–2800.

Kaare Brandt Petersen, Michael Syskind Pedersen, 2008. The matrix cookbook. Technical University of Denmark 7, 15 (2008), 510

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, 2018. Improving language understanding by generative pre-training. (2018)

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9

Mark J Schervish and Morris H DeGroot. 2012. Probability and statistics. Pearson Education

Uri Shalit, Fredrik D Johansson, and David Sontag. 2017. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning. PMLR, 3076–3085

Claudia Shi, David Blei, and Victor Veitch. 2019. Adapting Neural Networks for the Estimation of Treatment Effects. In Advances in Neural Information Processing Systems, H Wallach, H Larochelle, A Beygelzimer, F Alché-Buc, E Fox, and R Garnett (Eds.). Vol. 32. Curran Associates, Inc. [link].

Tian Wang, Yuri M Brovman, and Sriganesh Madhvanath. 2021. Personalized embedding-based e-commerce recommendations at ebay. arXiv preprint arXiv:2102.06156 (2021).

Kevin P Yancey and Burr Settles. 2020. A sleeping, recovering bandit algorithm for optimizing recurring notifications. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3008–3016.
PINTO, Victor M. Magalhães; OLIVEIRA, Leonardo M. Murtha; DE SOUSA, Ysabelle Pinheiro; DANYI, Stefania L.; LIMA, Dimas S.. Smart Notifications – An ML-based Framework to Boost User Engagement. In: SIMPÓSIO BRASILEIRO DE SISTEMAS MULTIMÍDIA E WEB (WEBMEDIA), 29. , 2023, Ribeirão Preto/SP. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2023 . p. 25–31.