Coordination within Conversational Agents with Multiple Sources

  • Vinícius Bitencourt Matos Universidade de São Paulo
  • Ricardo Grava Universidade de São Paulo
  • Rodrigo Tavares Universidade de São Paulo
  • Marcos Menon José Universidade de São Paulo
  • Paulo Pirozelli Universidade de São Paulo
  • Anarosa A. F. Brandão Universidade de São Paulo
  • Sarajane M. Peres Universidade de São Paulo
  • Fábio G. Cozman Universidade de São Paulo


Conversational agents can now operate with language models, rules, ontologies and varied other sources to provide smooth dialogue. However, the coordination of multiple sources in conversational agents is a challenge. We present a mechanism to effectively orchestrate multiple sources in a conversational agent, by relying on a client-server approach with an associated prompt generation module that deals with heterogeneous domain-oriented modules. As a detailed use case, we describe the architecture of a chatbot specialised in topics related to the Brazilian coast, and we study the benefits of our approach.

Palavras-chave: Dialogue agents, Prompt engineering, Natural language dialogue generation, Client-server architectures


Abreu, C.T.: Brazilian Coastal and Marine Protected Areas Importance, Current Status and Recommendations. Ph.D. thesis, Division for Ocean Affairs and the Law of the Sea, Office of Legal Affairs, The United Nations New York (2015)

Agência Nacional do Petróleo (ANP): Encarte de consolidação da produção 2021: Boletim da produção de petróleo e gás natural. [link]. (2021), online; accessed on 31st May 2022

Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., Do, Q.V., Xu, Y., Fung, P.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv:2302.04023 (2023)

Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are fewshot learners. arXiv:2005.14165 (2020)

Cação, F.N., José, M.M., Oliveira, A.S., Spindola, S., Costa, A.H.R., Cozman, F.G.: Deepagé: Answering questions in Portuguese about the Brazilian environment. In: Britto, A., Delgado, K.V. (eds.) Intelligent Systems - 10th Brazilian Conference, BRACIS 2021, Virtual Event, November 29 - December 3, 2021, Proceedings, Part II. Lecture Notes in Computer Science, vol. 13074, pp. 419–433. Springer (2021).

Carlos, G.O.: Desenvolvimento de um chatbot sobre a Amazônia Azul (2021), bachelor’s thesis written in Portuguese. Translated title: “Development of a Blue Amazon chatbot”.

Chase, H.: LangChain. [link] (2022)

Chen, D., Fisch, A., Weston, J., Bordes, A.: Reading Wikipedia to answer open-domain questions. In: Proc. of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1870–1879. Association for Computational Linguistics, Vancouver, Canada (2017).

Green, B.F., Wolf, A.K., Chomsky, C.L., Laughery, K.: Baseball: an automatic questionanswerer. In: IRE-AIEE-ACM ’61 (Western) (1961)

Jurafsky, D., Martin, J.H.: Speech and Language Processing. (draft), 3 edn. (2023)

Khatri, C., Venkatesh, A., Hedayatnia, B., Gabriel, R., Ram, A., Prasad, R.: Alexa prize—state of the art in conversational AI. AI Magazine 39(3), 40–55 (2018)

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive nlp tasks. In: Advances in Neural Information Processing Systems. vol. 33, pp. 9459–9474. Curran Associates, Inc. (2020)

OpenAI: Introducing ChatGPT. [link] (2022)

Ortiz, F.: The Blue Amazon, Brazil’s new natural resources frontier. [link]. (May 2015), [Online; accessed on 11th July 2022]

Pirozelli, P., Castro, A.B.R., de Oliveira, A.L.C., Oliveira, A.S., Cação, F.N., Silveira, I.C., Campos, J.G.M., Motheo, L.C., Figueiredo, L.F., Pellicer, L.F.A.O., José, M.A., José, M.M., de M. Ligabue, P., Grava, R.S., Tavares, R.M., Matos, V.B., Sym, Y.V., Costa, A.H.R., Brandão, A.A.F., Mauá, D.D., Cozman, F.G., Peres, S.M.: The BLue Amazon Brain (BLAB): A modular architecture of services about the Brazilian maritime territory. In: IJCAI Workshop: AI Modeling Oceans and Climate Change (AIMOCC 2022). pp. 1–11 (2022)

Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. arXiv:2302.04761 (2023)

Searle, J.R.: Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press, Cambridge, London (1969)

Shen, Y., Song, K., Tan, X., Li, D., Lu, W., Zhuang, Y.: HuggingGPT: Solving AI tasks with ChatGPT and its friends in HuggingFace. arXiv preprint arXiv:2303.17580 (2023), [link]

W3C World Wide Web Consortium: Web content accessibility guidelines 2.1, level a & level aa success criteria. [link] (Recommendation 05 June 2018)
MATOS, Vinícius Bitencourt; GRAVA, Ricardo; TAVARES, Rodrigo; JOSÉ, Marcos Menon; PIROZELLI, Paulo; BRANDÃO, Anarosa A. F.; PERES, Sarajane M.; COZMAN, Fábio G.. Coordination within Conversational Agents with Multiple Sources. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 20. , 2023, Belo Horizonte/MG. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2023 . p. 939-953. ISSN 2763-9061. DOI: