Skip to main content
SpringerLink
Log in
Book cover

Brazilian Conference on Intelligent Systems

BRACIS 2021: Intelligent Systems pp 104–120Cite as

Evaluating Topic Models in Portuguese Political Comments About Bills from Brazil’s Chamber of Deputies

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13074))

Abstract

The popular participation in Law-making is an important resource in the evolution of Democracy and Direct Legislation. The amount of legislative documents produced within the past decade has risen dramatically, making it difficult for law practitioners to attend to legislation and still listen to the opinion of the citizens. This work focuses on the use of topic models for summarizing and visualizing Brazilian comments about legislation (bills). In this paper, we provide a qualitative evaluation from a legal expert and compare it with the topics predicted by our model. For such, we designed a specific sentence embedding technique able to induce models for Portuguese texts, and we used these models as topic model, obtaining very good results. We experimentally compared our proposal with other techniques for multilingual sentence embeddings, evaluating them in three topical corpora prepared by us, two of them annotated by a specialist and the other automatically annotated by hashtags.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://twitter.com/.

  2. 2.

    https://forms.camara.leg.br/ex/enquetes/2209381.

  3. 3.

    https://forms.camara.leg.br/ex/enquetes/304008.

  4. 4.

    https://tfhub.dev/google/universal-sentence-encoder-multilingual-large/1.

  5. 5.

    https://www.wikipedia.org/.

  6. 6.

    https://nlp.stanford.edu/projects/snli/.

  7. 7.

    https://www.reddit.com/.

  8. 8.

    http://stackoverflow.com/.

  9. 9.

    https://help.yahoo.com/kb/answers.

  10. 10.

    Contrastive learning is a machine learning technique used to learn the general features of a dataset without labels by teaching the model which data points are similar or different [20].

  11. 11.

    Available at https://github.com/nadiafelix/Bracis2021.

  12. 12.

    https://github.com/MaartenGr/BERTopic.

  13. 13.

    https://github.com/princeton-nlp/SimCSE.

  14. 14.

    https://huggingface.co/.

  15. 15.

    To analyze the coherence in function of topics we enter with the number of topics.

  16. 16.

    We report the real Topics and respective predicted topics for PEC 471/2005 and Hashtag corpus in https://github.com/nadiafelix/Bracis2021.

References

  1. Abercrombie, G., Batista-Navarro, R.: Sentiment and position-taking analysis of parliamentary debates: a systematic literature review. J. Comput. Soc. Sci. 3(1), 245–270 (2020)

    Article  Google Scholar 

  2. Allahyari, M., et al.: A brief survey of text mining: classification, clustering and extraction techniques (2017)

    Google Scholar 

  3. Andrade, M.D.D., Rosa, B.D.C., Pinto, E.R.G.D.C.: Legal tech: analytics, inteligência artificial e as novas perspectivas para a prática da advocacia privada. Revista Direito GV 16(1) (2020)

    Google Scholar 

  4. Angelov, D.: Top2Vec: distributed representations of topics (2020). https://arxiv.org/abs/2008.09470

  5. Barberá, P., Rivero, G.: Understanding the political representativeness of Twitter users. Soc. Sci. Comput. Rev. 33(6), 712–729 (2015)

    Article  Google Scholar 

  6. Basu, S., Banerjee, A., Mooney, R.J.: Active semi-supervision for pairwise constrained clustering. In: Proceedings of the 2004 SIAM, International Conference on Data Mining. Society for Industrial and Applied Mathematics, April 2004

    Google Scholar 

  7. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  8. Campello, R.J.G.B., Moulavi, D., Zimek, A., Sander, J.: Hierarchical density estimates for data clustering, visualization, and outlier detection 10(1) (2015)

    Google Scholar 

  9. Chidambaram, M., et al.: Learning cross-lingual sentence representations via a multi-task dual-encoder model (2019)

    Google Scholar 

  10. Câmara dos Deputados: Dados Abertos da Câmara dos Deputados (Open Data of the Chamber of Deputies, when translated to English). https://dadosabertos.camara.leg.br (2021). Accessed 8 June 2021

  11. Câmara dos Deputados (Brazilian Chamber of Deputies, when translated to English): Enquetes (polls, when translated to English). https://www.camara.leg.br/enquetes. Accessed 5 May 2021

  12. Câmara dos Deputados (Brazilian Chamber of Deputies, when translated to English): Popular participation. https://www2.camara.leg.br/transparencia/servicos-ao-cidadao/participacao-popular (2021). Accessed 5 May 2021

  13. Cong, Y., Chen, B., Liu, H., Zhou, M.: Deep latent Dirichlet allocation with topic-layer-adaptive stochastic gradient Riemannian MCMC. In: Proceedings of the 34th International Conference on ML , vol. 70, pp. 864–873. ICML 2017, JMLR.org (2017)

    Google Scholar 

  14. Dale, R.: Law and word order: NLP in legal tech. Nat. Lang. Eng. 25(1), 211–217 (2019)

    Article  Google Scholar 

  15. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Comp. Linguistics: Human Language Tech, pp. 4171–4186. Minnesota, June 2019

    Google Scholar 

  16. Dieng, A.B., Ruiz, F.J.R., Blei, D.M.: Topic modeling in embedding spaces. Transactions of the Association for Computational Linguistics 8, 439–453 (2020)

    Article  Google Scholar 

  17. Evangelopoulos, N., Visinescu, L.: Text-mining the voice of the people. Commun. ACM 55(2), 62–69 (2012)

    Article  Google Scholar 

  18. Gandomi, A., Haider, M.: Beyond the hype: big data concepts, methods, and analytics. Int. J. Inf. Manage. 35(2), 137–144 (2015)

    Article  Google Scholar 

  19. Gao, T., Yao, X., Chen, D.: SimCSE: simple contrastive learning of sentence embeddings. https://arxiv.org/abs/2104.08821 (2021). Accessed 5 May 2021

  20. Giorgi, J., Nitski, O., Wang, B., Bader, G.: DeCLUTR: deep contrastive learning for unsupervised textual representations (2021)

    Google Scholar 

  21. Grootendorst, M.: BERTopic: leveraging BERT and c-TF-IDF to create easily interpretable topics. https://doi.org/10.5281/zenodo.4381785, note = Accessed 5 May 2021

  22. Harris, Z.: Word 10(2–3), 146–162 (1954)

    Google Scholar 

  23. Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 1999, pp. 50–57. Association for Computing Machinery, New York, NY, USA (1999)

    Google Scholar 

  24. Mahyoub, M., Hind, J., Woods, D., Wong, C., Hussain, A., Aljumeily, D.: Hierarchical text clustering and categorisation using a semi-supervised framework. In: 2019 12th International Conference on Developments in eSystems Engineering (DeSE). IEEE, October 2019

    Google Scholar 

  25. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013)

    Google Scholar 

  26. Moody, C.E.: Mixing Dirichlet topic models and word embeddings to make lda2vec (2016)

    Google Scholar 

  27. Nay, J.J.: Predicting and understanding law-making with word vectors and an ensemble model. PLoS ONE 12(5), e0176999 (2017)

    Google Scholar 

  28. Nebu, C.M., Joseph, S.: Semi-supervised clustering with soft labels. In: 2015 International Conference on Control Communication & Computing India (ICCC). IEEE, November 2015

    Google Scholar 

  29. Papadimitriou, C., Raghavan, P., Tamaki, H., Vempala, S.: Latent semantic indexing: a probabilistic analysis. J. Comput. Syst. Sci. 61(2), 217–235 (2000)

    Article  MathSciNet  Google Scholar 

  30. Parmelee, J.H., Bichard, S.L.: Politics and the Twitter Revolution: How Tweets Influence the Relationship Between Political Leaders and the Public. Lexington books, Lanham (2011)

    Google Scholar 

  31. Pavan, J.N.S., Pinochet, L.H.C., de Brelàz, G., dos Santos Júnior, D.L., Ribeiro, D.M.N.M.: Study of citizen engagement in the participation of elective mandate actions in the Brazilian legislature: analysis of the use of political techs. Cadernos EBAPE.BR 18(3), 525–542, September 2020

    Google Scholar 

  32. Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar, October 2014

    Google Scholar 

  33. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in NLP and the 9th International Joint Conference on NLP (EMNLP-IJCNLP), pp. 3982–3992. ACL, Hong Kong, China, November 2019

    Google Scholar 

  34. Reimers, N., Gurevych, I.: Making monolingual sentence embeddings multilingual using knowledge distillation. In: Proceedings of the 2020 Conference on Empirical Methods in NLP. ACL, November 2020

    Google Scholar 

  35. Röder, M., Both, A., Hinneburg, A.: Exploring the space of topic coherence measures. In: Proceedings of the 8a ACM International Conference on Web Search and Data Mining, pp. 399–408. WSDM 2015, New York, NY, USA (2015)

    Google Scholar 

  36. Sammut, C., Webb, G.I. (eds.): TF-IDF, pp. 986–987. Springer, US, Boston, MA (2010)

    Google Scholar 

  37. Souza, F., Nogueira, R., Lotufo, R.: BERTimbau: pretrained BERT models for Brazilian Portuguese. In: 9th Brazilian Conference on Intelligent Systems, BRACIS, Rio Grande do Sul, Brazil, 20–23 October 2020. (to appear)

    Google Scholar 

  38. Stevens, K., Kegelmeyer, P., Andrzejewski, D., Buttler, D.: Exploring topic coherence over many models and many topics. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 952–961. Association for Computational Linguistics, Jeju Island, Korea, July 2012

    Google Scholar 

  39. Sugathadasa, K., et al.: Legal document retrieval using document vector embeddings and deep learning. In: Arai, Kohei, Kapoor, Supriya, Bhatia, Rahul (eds.) SAI 2018. AISC, vol. 857, pp. 160–175. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-01177-2_12

    Chapter  Google Scholar 

  40. Thaiprayoon, S., Unger, H., Kubek, M.: Graph and centroid-based word clustering. In: Proceedings of the 4th International Conference on NLP and Information Retrieval. ACM, December 2020. https://doi.org/10.1145/3443279.3443290

  41. United Nations: Inclusion and more public participation, will help forge better government policies: Guterres. https://news.un.org/en/story/2020/09/1073742, September 2020. Accessed 5 May 2021

  42. Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: Is a correction for chance necessary? In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1073–1080. ICML 2009. Association for Computing Machinery, New York, NY, USA (2009)

    Google Scholar 

  43. Wallach, H., Murray, I., Salakhutdinov, R., Mimno, D.: Evaluation methods for topic models. In: Proceedings of the 26th Annual International Conference on ML (ICML 2009), pp. 1105–1112. ACM (2009)

    Google Scholar 

  44. Wang, K., Reimers, N., Gurevych, I.: TSDAE: using transformer-based sequential denoising auto-encoder for unsupervised sentence embedding learning (2021)

    Google Scholar 

  45. Willett, P.: Recent trends in hierarchic document clustering: a critical review. Inf. Process. Manage. 24(5), 577–597 (1988)

    Article  Google Scholar 

  46. Williams, A., Nangia, N., Bowman, S.: A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 1112–1122. Association for Computational Linguistics, New Orleans, Louisiana, June 2018

    Google Scholar 

  47. World Economic Forum: How much data is generated each day? https://www.weforum.org/agenda/2019/04/how-much-data-is-generated-each-day-cf4bddf29f/ (2019). Accessed 5 May 2021

  48. Yang, Y., et al.: Multilingual universal sentence encoder for semantic retrieval (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Mathematics and Computer Science, University of São Paulo, São Carlos, SP, 13566-590, Brazil

    Nádia F. F. da Silva, Marília Costa R. Silva, Fabíola S. F. Pereira, João Pedro M. Tarrega & João Vitor P. Beinotti

  2. Institute of Informatics, Federal University of Goiás, Goiânia, Go, Brazil

    Nádia F. F. da Silva

  3. Federal University of Uberlândia, Uberlândia, Brazil

    Fabíola S. F. Pereira

  4. Câmara dos Deputados - Palácio do Congresso Nacional - Praça dos Três Poderes Brasília, Brasília, DF, Brazil

    Márcio Fonseca, Francisco Edmundo de Andrade & André C. P. de L. F. de Carvalho

Authors
  1. Nádia F. F. da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marília Costa R. Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabíola S. F. Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. João Pedro M. Tarrega
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. João Vitor P. Beinotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Márcio Fonseca
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Francisco Edmundo de Andrade
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. André C. P. de L. F. de Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nádia F. F. da Silva .

Editor information

Editors and Affiliations

  1. Universidade Federal de Sergipe, São Cristóvão, Brazil

    André Britto

  2. Universidade de São Paulo, São Paulo, Brazil

    Karina Valdivia Delgado

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Silva, N.F.F.d. et al. (2021). Evaluating Topic Models in Portuguese Political Comments About Bills from Brazil’s Chamber of Deputies. In: Britto, A., Valdivia Delgado, K. (eds) Intelligent Systems. BRACIS 2021. Lecture Notes in Computer Science(), vol 13074. Springer, Cham. https://doi.org/10.1007/978-3-030-91699-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91699-2_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91698-5

  • Online ISBN: 978-3-030-91699-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation