DEEPAGÉ: Answering Questions in Portuguese About the Brazilian Environment

Abstract

The challenge of climate change and biome conservation is one of the most pressing issues of our time—particularly in Brazil, where key environmental reserves are located. Given the availability of large textual databases on ecological themes, it is natural to resort to question answering (QA) systems to increase social awareness and understanding about these topics. In this work, we introduce multiple QA systems that combine in novel ways the BM25 algorithm, a sparse retrieval technique, with PTT5, a pre-trained state-of-the-art language model. Our QA systems focus on the Portuguese language, thus offering resources not found elsewhere in the literature. As training data, we collected questions from open-domain datasets, as well as content from the Portuguese Wikipedia and news from the press. We thus contribute with innovative architectures and novel applications, attaining an F1-score of 36.2 with our best model.

F. N. Cação and M. M. José—These authors contributed equally to the work.

Appendices

7 Appendix A

To filter QA-pairs in PAQ, we created four sets of keywords: M (from “Must have”: “brazil*” and other states names), G (from “Good to have”; e.g. “deforestation”), U (from “Unique expressions”; e.g. “ibama” or “amazon rainforest”) and E (from “to Exclude”; e.g. “soccer”). For a pair to be selected, it must contain, in its question or answer, at least one expression belonging to M or U. If it was from U, it would already be selected; if it was from M, it could not contain any E expressions. Finally, if the QA-pair contains any keyword from G, it should also contain at least one from M, but none from E either.

8 Appendix B

We performed queries associated with the following keywords (here, translated into English): Amazon rainforest, Cerrado, Climate change, Deforestation, Environmental Conduct Adjustment Agreement, Extractive (from “Extractivism”), Forest fires, Funding for conservation, Green economy, Land grabbing, Mining and Protected areas. It should be noted that the decision to exclude some words is only understandable in Portuguese, since there is no reasonable parallel in English, as with the word “fist” for the keyword “Cerrado” - in Portuguese, the expression “punho cerrado” (“clenched fist”) is common, but it is clearly not directly related to the Cerrado biome. Also for more significant results, we eliminated news related to “agrobusiness” in the Green economy keyword and “militias” in the Land grabbing one.

9 Appendix C

To perform the experiments, we leverage a machine with an AMD Ryzen 9 3950X Processor with 32 CPUs, 64 GB of RAM, 2 NVIDIA RTX 3090 GPUs of 24GB each, on an Ubuntu 20 LTS. Under these conditions, the training of each model lasted between 5h and 8h, with the best model being trained in about 7h40. The inference time – that is, the time it took to a model to generate an answer to a single question from the test set – was around 0.006s.