Assessing Quality Through Use: A Usability-Based Evaluation of the Conta gov.br Authentication Platform
Resumo
This article presents an empirical study that evaluated the Conta gov.br authentication platform, a nationally adopted service in Brazil, focusing on usability as a dimension of software quality. Thirty participants took part in usability testing using standardized metrics: System Usability Scale (SUS), Single Ease Question (SEQ), and Task Completion Time (TCT). Results showed variations in perceived usability influenced by demographic factors such as age and education level. Statistical correlations between scores and user profiles were also analyzed. Based on the findings, a Practical Guide was developed to support public institutions in their adoption strategies. The results contribute evidence-based insights to inform quality-oriented decisions in digital public services.
Palavras-chave:
usability, authentication, public digital services, software quality, Conta gov.br
Referências
Alexandre Aragão, Lucas Machado, Nathyane Moreno, Davi Viana, Francisco Silva, Tércio Sousa, Luis Rivero, Ariel Teles, Arlindo F. Da Conceição, and Inaldo Costa. 2020. Evoluindo uma aplicação para Cidades Inteligentes através de Avaliação de Métricas de Qualidade e Usabilidade. iSys - Brazilian Journal of Information Systems 13, 3 (June 2020), 55–81. DOI: 10.5753/isys.2020.795
Aaron Bangor, Phil Kortum, and James Miller. 2009. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Stud. 4 (April 2009), 114–123. [link]
J.M. Christian Bastien. 2010. Usability testing: a review of some methodological and technical aspects of the method. International Journal of Medical Informatics 79, 4 (April 2010), e18–e23. DOI: 10.1016/j.ijmedinf.2008.12.004
Nigel Bevan and Miles Macleod. 1994. Usability measurement in context. Behaviour & Information Technology 13, 1-2 (Jan. 1994), 132–145. DOI: 10.1080/01449299408914592
Jonas Blattgerste, Jan Behrends, and Thies Pfeiffer. 2022. A Web-Based Analysis Toolkit for the System Usability Scale. In Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments (Corfu, Greece) (PETRA ’22). Association for Computing Machinery, New York, NY, USA, 237–246. DOI: 10.1145/3529190.3529216
Kenneth S. Bordens and Bruce B. Abbott. 2011. Research design and methods: a process approach (8th ed ed.). McGraw-Hill, New York. OCLC: ocn436028182.
John Brooke. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind. 189 (11 1995).
Jacob Cohen. 2013. Statistical Power Analysis for the Behavioral Sciences (2nd ed ed.). Taylor and Francis, Hoboken.
Edward E. Cureton. 1956. Rank-Biserial Correlation. Psychometrika 21, 3 (Sept. 1956), 287–290. DOI: 10.1007/BF02289138
Fred D. Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13, 3 (Sept. 1989), 319. DOI: 10.2307/249008
Demétrius de Almeida Jubé. 2025. Guia Prático para Adoção da Conta gov.br em Serviços Públicos Digitais. [link]. Disponível em: GitHub. Acesso em: out. 2025..
Secretaria de Governo Digital. 2024. Roteiro de Integração do Login Único — Roteiro de Integração do Login Único documentation. [link]
Departamento de Serviços Públicos Digitais. 2019. Por quem foi criado? — Dúvidas Frequentes da Conta gov.br. Disponível em: [link]. Acesso em: 13 mai. 2025.
Nebojsa Djordjevic. 2017. Usability: Key characteristic of software quality. Vojnotehnicki glasnik 65, 2 (2017), 513–529. DOI: 10.5937/vojtehg65-11028 Publisher: Centre for Evaluation in Education and Science (CEON/CEES).
Governo Federal do Brasil. 2021. Termo de Uso e Política de Privacidade - gov.br. Disponível em: [link]. Acesso em: 13 mai. 2025.
Governo Federal do Brasil. 2023. Estratégia de Governo Digital 2020–2022. Disponível em: [link]. Acesso em: 13 mai. 2025.
Governo Federal do Brasil. 2024. Serviço de Integração aos Produtos de Identidade Digital gov.br. [link]
Governo Federal do Brasil. 2025. Onde usar a conta gov.br. [link]
Senado Federal. 2023. Conheça os resultados e o funcionamento do Portal e-Cidadania. Technical Report. [link]
Sinval Ferreira Resende Júnior and Leticia Lopes Leite. 2024. Um estudo sobre problemas de usabilidade no software de processos administrativos eletrônicos do Governo Federal do Brasil. iSys - Brazilian Journal of Information Systems 17, 1 (June 2024). DOI: 10.5753/isys.2024.4141
Demétrius Jubé. 2025. Assessing Quality Through Use: A Usability-Based Evaluation of the Conta gov.br Authentication Platform - Material da pesquisa. DOI: 10.6084/M9.FIGSHARE.29631566 Artwork Size: 443910 Bytes Pages: 443910 Bytes.
Demétrius de Almeida Jubé and Letícia Lopes Leite. 2024. Usability in Single Sign-On Solutions: What to Evaluate? DOI: 10.6084/m9.figshare.30278305. Peer-reviewed paper accepted at CISTI 2024 (19th Iberian Conference on Information Systems and Technologies), Salamanca, Spain. Available in open access due to pending publication of conference proceedings.
Maina Korir, Simon Parkin, and Paul Dunphy. 2022. An Empirical Study of a Decentralized IdentityWallet: Usability, Security, and Perspectives on User Control. In Proceedings of the 18th Symposium on Usable Privacy and Security, SOUPS 2022. 195 – 211. [link] Type: Conference paper.
Kwang Bok Lee and R.A. Grice. 2004. Developing a new usability testing method for mobile devices. In International Professional Communication Conference, 2004. IPCC 2004. Proceedings. IEEE, Minneapolis, MN, USA, 115–127. DOI: 10.1109/IPCC.2004.1375285
Douglas Fabiano Lourenço, Elenice Carmona Valentim, and Maria Helena Baena de Moraes Lopes. 2022. Tradução e adaptação transcultural da System Usability Scale para o português do Brasil. Aquichan 22, 2 (maio 2022), e2228. DOI: 10.5294/aqui.2022.22.2.8
Mateus De Souza Monteiro, Gabriely Oliveira Da Silva Batista, and Luciana Cardoso De Castro Salgado. 2023. Investigating usability pitfalls in Brazilian and Foreign governmental chatbots. Journal on Interactive Systems 14, 1 (July 2023), 331–340. DOI: 10.5753/jis.2023.3104
Iasonas Polakis, Panagiotis Ilia, Federico Maggi, Marco Lancini, Georgios Kontaxis, Stefano Zanero, Sotiris Ioannidis, and Angelos D. Keromytis. 2014. Faces in the Distorting Mirror: Revisiting Photo-Based Social Authentication. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security (CCS ’14). Association for Computing Machinery, New York, NY, USA, 501–512. DOI: 10.1145/2660267.2660317 event-place: Scottsdale, Arizona, USA.
Jenny Preece, Yvonne Rogers, and Helen Sharp. 2002. Interaction design: beyond human-computer interaction. J. Wiley & Sons, New York, NY.
Jeffrey Rubin and Dana Chisnell. 2008. Handbook of usability testing: how to plan, design, and conduct effective tests (2nd ed ed.). Wiley Pub, Indianapolis, IN. OCLC: ocn212204392.
M. Angela Sasse, Michelle Steves, Kat Krol, and Dana Chisnell. 2014. The Great Authentication Fatigue – And How to Overcome It. In Cross-Cultural Design, David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kleinberg, Alfred Kobsa, Friedemann Mattern, John C. Mitchell, Moni Naor, Oscar Nierstrasz, C. Pandu Rangan, Bernhard Steffen, Demetri Terzopoulos, Doug Tygar, Gerhard Weikum, and P. L. Patrick Rau (Eds.). Vol. 8528. Springer International Publishing, Cham, 228–239. DOI: 10.1007/978-3-319-07308-8_23 Series Title: Lecture Notes in Computer Science.
Jeff Sauro and James R. Lewis. 2016. Quantifying the user experience: practical statistics for user research (2nd edition ed.). Elsevier, Morgan Kaufmann, Amsterdam Boston Heidelberg.
Patrick Schober, Christa Boer, and Lothar A. Schwarte. 2018. Correlation Coefficients: Appropriate Use and Interpretation. Anesthesia & Analgesia 126, 5 (May 2018), 1763–1768. DOI: 10.1213/ANE.0000000000002864
S. S. Shapiro and M. B. Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3-4 (Dec. 1965), 591–611. DOI: 10.1093/biomet/52.3-4.591
San-Tsai Sun, Eric Pospisil, Ildar Muslukhov, Nuray Dindar, Kirstie Hawkey, and Konstantin Beznosov. 2013. Investigating Users’ Perspectives of Web Single Sign-On: Conceptual Gaps and Acceptance Model. ACM Transactions on Internet Technology 13, 1 (Nov. 2013), 1–35. DOI: 10.1145/2532639
Donna P Tedesco and Thomas S Tullis. 2006. A Comparison of Methods for Eliciting Post-Task Subjective Ratings in Usability Testing. (2006).
Thomas Tullis and Jacqueline Stetson. 2006. A Comparison of Questionnaires for Assessing Website Usability. (June 2006).
Raphael Vallat. 2018. Pingouin: statistics in Python. Journal of Open Source Software 3, 31 (Nov. 2018), 1026. DOI: 10.21105/joss.01026
José Vieira and Rodrigo Andrade. 2024. On the investigation of Usability in the Caixa Tem Application. Journal on Interactive Systems 15, 1 (May 2024), 417–433. DOI: 10.5753/jis.2024.3889
Tianhao Wang, Huangyi Ge, Omar Chowdhury, Hemanta K. Maji, and Ninghui Li. 2016. On the Security and Usability of Segment-based Visual Cryptographic Authentication Protocols. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, Vienna Austria, 603–615. DOI: 10.1145/2976749.2978417
Aaron Bangor, Phil Kortum, and James Miller. 2009. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Stud. 4 (April 2009), 114–123. [link]
J.M. Christian Bastien. 2010. Usability testing: a review of some methodological and technical aspects of the method. International Journal of Medical Informatics 79, 4 (April 2010), e18–e23. DOI: 10.1016/j.ijmedinf.2008.12.004
Nigel Bevan and Miles Macleod. 1994. Usability measurement in context. Behaviour & Information Technology 13, 1-2 (Jan. 1994), 132–145. DOI: 10.1080/01449299408914592
Jonas Blattgerste, Jan Behrends, and Thies Pfeiffer. 2022. A Web-Based Analysis Toolkit for the System Usability Scale. In Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments (Corfu, Greece) (PETRA ’22). Association for Computing Machinery, New York, NY, USA, 237–246. DOI: 10.1145/3529190.3529216
Kenneth S. Bordens and Bruce B. Abbott. 2011. Research design and methods: a process approach (8th ed ed.). McGraw-Hill, New York. OCLC: ocn436028182.
John Brooke. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind. 189 (11 1995).
Jacob Cohen. 2013. Statistical Power Analysis for the Behavioral Sciences (2nd ed ed.). Taylor and Francis, Hoboken.
Edward E. Cureton. 1956. Rank-Biserial Correlation. Psychometrika 21, 3 (Sept. 1956), 287–290. DOI: 10.1007/BF02289138
Fred D. Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13, 3 (Sept. 1989), 319. DOI: 10.2307/249008
Demétrius de Almeida Jubé. 2025. Guia Prático para Adoção da Conta gov.br em Serviços Públicos Digitais. [link]. Disponível em: GitHub. Acesso em: out. 2025..
Secretaria de Governo Digital. 2024. Roteiro de Integração do Login Único — Roteiro de Integração do Login Único documentation. [link]
Departamento de Serviços Públicos Digitais. 2019. Por quem foi criado? — Dúvidas Frequentes da Conta gov.br. Disponível em: [link]. Acesso em: 13 mai. 2025.
Nebojsa Djordjevic. 2017. Usability: Key characteristic of software quality. Vojnotehnicki glasnik 65, 2 (2017), 513–529. DOI: 10.5937/vojtehg65-11028 Publisher: Centre for Evaluation in Education and Science (CEON/CEES).
Governo Federal do Brasil. 2021. Termo de Uso e Política de Privacidade - gov.br. Disponível em: [link]. Acesso em: 13 mai. 2025.
Governo Federal do Brasil. 2023. Estratégia de Governo Digital 2020–2022. Disponível em: [link]. Acesso em: 13 mai. 2025.
Governo Federal do Brasil. 2024. Serviço de Integração aos Produtos de Identidade Digital gov.br. [link]
Governo Federal do Brasil. 2025. Onde usar a conta gov.br. [link]
Senado Federal. 2023. Conheça os resultados e o funcionamento do Portal e-Cidadania. Technical Report. [link]
Sinval Ferreira Resende Júnior and Leticia Lopes Leite. 2024. Um estudo sobre problemas de usabilidade no software de processos administrativos eletrônicos do Governo Federal do Brasil. iSys - Brazilian Journal of Information Systems 17, 1 (June 2024). DOI: 10.5753/isys.2024.4141
Demétrius Jubé. 2025. Assessing Quality Through Use: A Usability-Based Evaluation of the Conta gov.br Authentication Platform - Material da pesquisa. DOI: 10.6084/M9.FIGSHARE.29631566 Artwork Size: 443910 Bytes Pages: 443910 Bytes.
Demétrius de Almeida Jubé and Letícia Lopes Leite. 2024. Usability in Single Sign-On Solutions: What to Evaluate? DOI: 10.6084/m9.figshare.30278305. Peer-reviewed paper accepted at CISTI 2024 (19th Iberian Conference on Information Systems and Technologies), Salamanca, Spain. Available in open access due to pending publication of conference proceedings.
Maina Korir, Simon Parkin, and Paul Dunphy. 2022. An Empirical Study of a Decentralized IdentityWallet: Usability, Security, and Perspectives on User Control. In Proceedings of the 18th Symposium on Usable Privacy and Security, SOUPS 2022. 195 – 211. [link] Type: Conference paper.
Kwang Bok Lee and R.A. Grice. 2004. Developing a new usability testing method for mobile devices. In International Professional Communication Conference, 2004. IPCC 2004. Proceedings. IEEE, Minneapolis, MN, USA, 115–127. DOI: 10.1109/IPCC.2004.1375285
Douglas Fabiano Lourenço, Elenice Carmona Valentim, and Maria Helena Baena de Moraes Lopes. 2022. Tradução e adaptação transcultural da System Usability Scale para o português do Brasil. Aquichan 22, 2 (maio 2022), e2228. DOI: 10.5294/aqui.2022.22.2.8
Mateus De Souza Monteiro, Gabriely Oliveira Da Silva Batista, and Luciana Cardoso De Castro Salgado. 2023. Investigating usability pitfalls in Brazilian and Foreign governmental chatbots. Journal on Interactive Systems 14, 1 (July 2023), 331–340. DOI: 10.5753/jis.2023.3104
Iasonas Polakis, Panagiotis Ilia, Federico Maggi, Marco Lancini, Georgios Kontaxis, Stefano Zanero, Sotiris Ioannidis, and Angelos D. Keromytis. 2014. Faces in the Distorting Mirror: Revisiting Photo-Based Social Authentication. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security (CCS ’14). Association for Computing Machinery, New York, NY, USA, 501–512. DOI: 10.1145/2660267.2660317 event-place: Scottsdale, Arizona, USA.
Jenny Preece, Yvonne Rogers, and Helen Sharp. 2002. Interaction design: beyond human-computer interaction. J. Wiley & Sons, New York, NY.
Jeffrey Rubin and Dana Chisnell. 2008. Handbook of usability testing: how to plan, design, and conduct effective tests (2nd ed ed.). Wiley Pub, Indianapolis, IN. OCLC: ocn212204392.
M. Angela Sasse, Michelle Steves, Kat Krol, and Dana Chisnell. 2014. The Great Authentication Fatigue – And How to Overcome It. In Cross-Cultural Design, David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kleinberg, Alfred Kobsa, Friedemann Mattern, John C. Mitchell, Moni Naor, Oscar Nierstrasz, C. Pandu Rangan, Bernhard Steffen, Demetri Terzopoulos, Doug Tygar, Gerhard Weikum, and P. L. Patrick Rau (Eds.). Vol. 8528. Springer International Publishing, Cham, 228–239. DOI: 10.1007/978-3-319-07308-8_23 Series Title: Lecture Notes in Computer Science.
Jeff Sauro and James R. Lewis. 2016. Quantifying the user experience: practical statistics for user research (2nd edition ed.). Elsevier, Morgan Kaufmann, Amsterdam Boston Heidelberg.
Patrick Schober, Christa Boer, and Lothar A. Schwarte. 2018. Correlation Coefficients: Appropriate Use and Interpretation. Anesthesia & Analgesia 126, 5 (May 2018), 1763–1768. DOI: 10.1213/ANE.0000000000002864
S. S. Shapiro and M. B. Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3-4 (Dec. 1965), 591–611. DOI: 10.1093/biomet/52.3-4.591
San-Tsai Sun, Eric Pospisil, Ildar Muslukhov, Nuray Dindar, Kirstie Hawkey, and Konstantin Beznosov. 2013. Investigating Users’ Perspectives of Web Single Sign-On: Conceptual Gaps and Acceptance Model. ACM Transactions on Internet Technology 13, 1 (Nov. 2013), 1–35. DOI: 10.1145/2532639
Donna P Tedesco and Thomas S Tullis. 2006. A Comparison of Methods for Eliciting Post-Task Subjective Ratings in Usability Testing. (2006).
Thomas Tullis and Jacqueline Stetson. 2006. A Comparison of Questionnaires for Assessing Website Usability. (June 2006).
Raphael Vallat. 2018. Pingouin: statistics in Python. Journal of Open Source Software 3, 31 (Nov. 2018), 1026. DOI: 10.21105/joss.01026
José Vieira and Rodrigo Andrade. 2024. On the investigation of Usability in the Caixa Tem Application. Journal on Interactive Systems 15, 1 (May 2024), 417–433. DOI: 10.5753/jis.2024.3889
Tianhao Wang, Huangyi Ge, Omar Chowdhury, Hemanta K. Maji, and Ninghui Li. 2016. On the Security and Usability of Segment-based Visual Cryptographic Authentication Protocols. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, Vienna Austria, 603–615. DOI: 10.1145/2976749.2978417
Publicado
04/11/2025
Como Citar
JUBÉ, Demétrius de Almeida; LEITE, Letícia Lopes.
Assessing Quality Through Use: A Usability-Based Evaluation of the Conta gov.br Authentication Platform. In: SIMPÓSIO BRASILEIRO DE QUALIDADE DE SOFTWARE (SBQS), 24. , 2025, São José dos Campos/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 184-192.
DOI: https://doi.org/10.5753/sbqs.2025.14871.
