Automating Risk of Bias Inference in Clinical Studies - Dias, Abel C.; Moreira, Viviane P.; Comba, João Luiz D. (Anais Estendidos do Simpósio Brasileiro de Computação Aplicada à Saúde (SBCAS); 2025: Anais Estendidos do XXV Simpósio Brasileiro de Computação Aplicada à SaúdeDO - 10.5753/sbcas_estendido.2025.7404, 2025)
One of the best quality indicators in the clinical domain is the risk of bias (RoB). Bias refers to any systematic error in results that may lead to misinterpretation. In the systematic review process, human reviewers manually assess the RoB. Existing works attempt to automate this process using support vector machines (SVM), convolutional neural networks (CNN), or logistic regression. To the best of our knowledge, no previous work has explored Transformer-based models for the RoB assessment in clinical studies. In this work, we propose a novel model for RoB inference based on the Transformers architecture, called RoBIn (i.e., Risk of Bias Inference). We employ a machine reading comprehension (MRC) approach to extract evidence that is then classified with a RoB label. Furthermore, we use distant supervision to annotate a dataset for MRC and RoB inference. As a final contribution, a large language model (LLM) application was created to receive clinical trials as input and to assess the RoB. The proposed model outperforms state-of-the-art approaches and other LLMs in many settings, with high accuracy (AUC-ROC= 0.83) for different bias types.