Can AI Judge Usability? A Comparative Analysis of Generative Tools on Climate Conference Websites - Lima, Danilo Teixeira; Medeiros, Alana M.; Paulino, Rita de Cássia R.; Seruffo, Marcos César R. (Anais do Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC); 2025: Anais do XXIV Simpósio Brasileiro sobre Fatores Humanos em Sistemas ComputacionaisDO - 10.5753/ihc.2025.10805, 2025)
Introduction: This paper analyzes the usability of COP30 (Brazil) and COP29 (Azerbaijan) websites using four Generative Artificial Intelligences (GAIs): ChatGPT, DeepSeek, Gemini, and Microsoft 365 Copilot. Objective: To examine the role of GAIs as support tools in usability evaluations. Methodology: Following ISO 9241-210, each GAI received a standardized prompt, and outputs were analyzed through Directed Categorical Content Analysis. Results: GAIs identified issues such as deep navigation, absent accessibility mechanisms, and terminological inconsistencies, but showed limits in cultural sensitivity and technical accessibility. The study highlights opportunities and cautions for their application in real projects.