Opportunities vs. Risks: Exploring Automatic Annotation of Financial Polarity Biases via Large Language Models

• Viviane Romero Universidade Federal Fluminense
• Gabriel Assis Universidade Federal Fluminense
• Jonnathan Carvalho Instituto Federal Fluminense
• Paulo Mann Universidade Federal do Rio de Janeiro
• Aline Paes Universidade Federal Fluminense

DOI: https://doi.org/10.5753/kdmile.2025.247742

Resumo

The financial market encompasses investors with distinct risk profiles - conservative, moderate and aggressive — each reflecting different attitudes toward gains and losses. These profiles influence the interpretation of financial news, particularly regarding the comparison between the labels "positive"/"negative" and the categories "opportunity"/"risk". Although these pairs of terms may initially appear equivalent, their practical application reveals notable inconsistencies. This paper employs large language models (LLMs) to annotate financial news, investigating whether such models capture the biases associated with each investor profile. We analyze the correlation between "opportunity" and "positive" and between "risk" and "negative" in the labels generated by the models. Furthermore, we examine whether, in the absence of explicit instructions regarding risk preference, the LLMs implicitly adopt a default bias when performing sentiment analysis on financial texts. Our findings provide insights into how risk profiles influence model behavior and suggest directions for improving both the personalization and accuracy of polarity detection in financial news analysis.

Palavras-chave: financial polarity, large language models, prompt engineering

Referências

Ahmed, T., Devanbu, P., Treude, C., and Pradel, M. Can LLMs replace manual annotation of software engineering artifacts?, 2024.

Anbaee Farimani, S., Vafaei Jahan, M., Milani Fard, A., and Tabbakh, S. R. K. Investigating the informativeness of technical indicators and news sentiment in financial market price prediction. Knowledge-Based Systems vol. 247, pp. 108742, 2022.

Ardekani, A. M., Bertz, J., Bryce, C., Dowling, M., and Long, S. C. Finsentgpt: A universal financial sentiment engine? International Review of Financial Analysis vol. 94, pp. 103291, 2024.

Bashar Yaser Almansour, S. E. and Almansour, A. Y. Behavioral finance factors and investment decisions: A mediating role of risk perception. Cogent Economics & Finance 11 (2): 2239032, 2023.

Chen, C.-C., Tseng, Y.-M., Kang, J., Lhuissier, A., Seki, Y., Day, M.-Y., Tu, T.-T., and Chen, H.-H. Multi-lingual ESG impact type identification. In Proceedings of the Sixth Workshop on Financial Technology and Natural Language Processing, C.-C. Chen, H.-H. Huang, H. Takamura, H.-H. Chen, H. Sakaji, and K. Izumi (Eds.). Association for Computational Linguistics, Bali, Indonesia, pp. 46–50, 2023

Day, M.-Y. and Lee, C.-C. Deep learning for financial sentiment analysis on finance news providers. In 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). pp. 1127–1134, 2016.

Dickason, Z. and Ferreira, S. Establishing a link between risk tolerance, investor personality and behavioural finance in south africa. Cogent Economics & Finance 6 (1): 1519898, 2018.

Dong, X., Wang, S., Lin, D., Rajbahadur, G. K., Zhou, B., Liu, S., and Hassan, A. E. Promptexp: Multi-granularity prompt explanation of large language models, 2024.

Gemini-Team. Gemini: A family of highly capable multimodal models, 2024.

Gemma Team. Gemma 3: Open models from google deepmind. [link], 2025. Modelo open-weight de 27 bilhões de parâmetros com capacidades multimodais.

Google DeepMind Team. Gemini 2.5 flash model card. [link], 2025. Modelo Gemini 2.5 Flash com foco em alta performance e baixa latência.

Heston, S. and Sinha, N. News vs. sentiment: Predicting stock returns from news stories. Financial Analysts Journal vol. 73, pp. 1–17, 06, 2017.

Hiew, J. Z. G., Huang, X., Mou, H., Li, D., Wu, Q., and Xu, Y. Bert-based financial sentiment index and lstm-based stock return predictability, 2022.

Khadjeh Nassirtoussi, A., Aghabozorgi, S., Ying Wah, T., and Ngo, D. C. L. Text mining for market prediction: A systematic review. Expert Systems with Applications 41 (16): 7653–7670, 2014.

Krippendorff, K. Computing krippendorff’s alpha-reliability, 2011.

Loughran, T. When is a liability not a liability? textual analysis, dictionaries, and 10-ks. The Journal of Finance vol. 66, pp. 35 – 65, 02, 2011.

Mak, M. and Ip, W. An exploratory study of investment behaviour of investors. International Journal of Engineering Business Management vol. 9, pp. 184797901771152, 06, 2017.

Man, X., Luo, T., and Lin, J. Financial sentiment analysis(fsa): A survey. 2019 IEEE International Conference on Industrial Cyber Physical Systems (ICPS), 2019.

Nguyen, L., Gallery, G., and Newton, C. The joint influence of financial risk perception and risk tolerance on individual investment decision-making. Accounting & Finance vol. 59, 09, 2017.

OpenAI-Team. Gpt-4 technical report, 2024.

Pangakis, N. and Wolken, S. Knowledge distillation in automated annotation: Supervised text classification with LLM-generated training labels, 2024.

Pompian, M. Risk profiling through a behavioral finance lens. CFA Institute Research Foundation, 2016.

Saad, S. and Saberi, B. Sentiment analysis or opinion mining: A review. International Journal on Advanced Science, Engineering and Information Technology vol. 7, pp. 1660, 10, 2017.

Tan, Z., Li, D., Wang, S., Beigi, A., Jiang, B., Bhattacharjee, A., Karami, M., Li, J., Cheng, L., and Liu, H. Large language models for data annotation and synthesis: A survey, 2024.

United-Nations. Transforming our world: The 2030 agenda for sustainable development, 2015. Accessed: 2025-02-02.

UNPRI. Principles for responsible investment, 2006. Accessed: 2025-02-02.

Wang, Y., Stevens, D., Shah, P., Jiang, W., Liu, M., Chen, X., Kuo, R., Li, N., Gong, B., Lee, D., Hu, J., Zhang, N., and Kamma, B. Model-in-the-loop (milo): Accelerating multimodal AI data annotation with LLMs, 2024.

White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., and Schmidt, D. C. A prompt pattern catalog to enhance prompt engineering with ChatGPT. arXiv preprint arXiv:2302.11382 , 2023.

Wu, J., Wang, X., and Jia, W. Enhancing text annotation through rationale-driven collaborative few-shot prompting, 2024.

Yadav, S., Choppa, T., and Schlechtweg, D. Towards automating text annotation: A case study on semantic proximity annotation using GPT-4, 2024.

Yang, A., Liu, W., Zhu, B., et al. Qwen3 technical report. [link], 2025. Descrição técnica do modelo Qwen-3-32B, um grande modelo de linguagem com 32 bilhões de parâmetros.

Yang, S., Rosenfeld, J., and Makutonin, J. Financial aspect-based sentiment analysis using deep representations. CoRR vol. abs/1808.07931, 2018.

Zhang, R., Li, Y., Ma, Y., Zhou, M., and Zou, L. LLMaAA: Making large language models as active annotators. In Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali (Eds.). Association for Computational Linguistics, Singapore, pp. 13088–13103, 2023.