Large Language Models for Structured Chest CT Reporting in Portuguese: A Comparative Study with Radiologist Validation - Petruceli, Juliana; Oliveira, Marcelo; Ferreira, Tarcisio; Sabino, Jose Arthur (Anais do Simpósio Brasileiro de Computação Aplicada à Saúde (SBCAS); 2026: Anais do XXVI Simpósio Brasileiro de Computação Aplicada à Saúde, 2026)
To evaluate LLMs for converting free-text Portuguese chest CT reports into structured JSON for clinical communication and data reuse. Gemini 1.5 Flash, GPT-4o, and LLaMA 3.3 were tested on 1,102 de-identified reports using a dynamic JSON template with few-shot prompting. Validation combined radiologist review and quantitative metrics. All models produced coherent structured outputs. Gemini achieved the best agreement (macro-F1 0.852; micro-F1 0.853), followed by LLaMA (0.806; 0.809) and GPT-4o (0.797;0.798). LLM-assisted structuring of Portuguese chest CT reports is feasible and attains high agreement with manual references; section-aware prompting and JSON validation improve robustness.