LLM vs. LLM: uso de red team e AML para avaliar a segurança de LLMs
Resumo
A cibersegurança apresenta um desafio complexo no que tange proteção relacionada à Inteligência Artificial (IA). Esta área também propõe segurança no envolvimento de tecnologia, dados e indivíduos. Nesse sentido, envolvida à cibersegurança apresenta-se uma área chamada Adversarial Machine Learning (AML) que se aprofunda no estudo e desenvolvimento de ferramentas para proteção de inteligentes sistemas baseados no aprendizado de máquinas. Estudos tem sido realizados no âmbito de AML, muito embora com resultados de pesquisas direcionadas, geralmente, para um ou dois tipos de LLMs. Este artigo apresenta uma proposta para se aprofundar nos três principais LLMs (GPT-4, Google Gemini e LlaMA) e apresentar um método de identificação de ameaças utilizando Large Language Models (LLMs) contra LLMs para soluções baseadas em testes que visam mitigar a exploração de sistemas de maneira maliciosa.
Palavras-chave:
Red Team, cibersegurança, AML, IA adversarial, grandes modelos de línguas
Referências
Brookson, C. et al. Definition of cybersecurity-gaps and overlaps in standardisation. Heraklion, ENISA, 2015.
Vassilev, A. et al. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. Apostol Vassilev, Alina Oprea, Alie Fordyce, Hyrum Andersen, 2024.
Stanton, B.; Jensen, T. Trust and Artificial Intelligence. en. NIST Interagency/Internal Report (NISTIR), National Institute of Standards e Technology, Gaithersburg, MD, mar. 2021. Disponível em: [link].
European Union Agency for Cybersecurity (ENISA). Securing Machine Learning Algorithms. 2021. [link]. ISBN: 978-92-9204-543-2, DOI: 10.2824/874249, Catalogue Nr.: TP-06-21-153-EN-N.
MITRE. MITRE ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems). Visitado em Dezembro de 2023. 2023. Disponível em: [link].
Xu, J. et al. Bot-Adversarial Dialogue for Safe Conversational Agents. In: Toutanova, K. et al. (Ed.). Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, jun. 2021. P. 2950–2968. DOI: 10.18653/v1/2021.naacl-main.235.
Li, Y.; Liu, Q. A comprehensive review study of cyber-attacks and cyber security; Emerging trends and recent developments. Energy Reports, v. 7, p. 8176–8186, 2021. ISSN 2352-4847. DOI: 10.1016/j.egyr.2021.08.126.
Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems, v. 30, 2017.
Yao, Y. et al. A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly. High-Confidence Computing, v. 4, n. 2, p. 100211, 2024. ISSN 2667-2952. DOI: 10.1016/j.hcc.2024.100211.
Biggio, B.; Nelson, B.; Laskov, P. Poisoning Attacks against Support Vector Machines. 2013. arXiv: 1206.6389 [cs.LG]. Disponível em: [link].
Kurita, K.; Michel, P.; Neubig, G.Weight Poisoning Attacks on Pre-trained Models. 2020. Visitado em Outubro de 2024. Disponível em: [link].
Ge, S. et al. MART: Improving LLM Safety with Multi-round Automatic Red-Teaming. 2023. arXiv: 2311.07689 [cs.CL]. Disponível em: [link].
Wan, A. et al. Poisoning Language Models During Instruction Tuning. 2023. Visitado em Outubro de 2024. Disponível em: [link].
Zhang, Y. et al. Siren’s song in the AI ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219, 2023.
Ding, X. et al. HPC-GPT: Integrating Large Language Model for High-Performance Computing. In: PROCEEDINGS of the SC ’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis. Denver, CO, USA: Association for Computing Machinery, 2023. (SC-W ’23), p. 951–960. ISBN 9798400707858. DOI: 10.1145/3624062.3624172.
Brown, T. B. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.
Liang, P. et al. Holistic Evaluation of Language Models. 2023. arXiv: 2211. 09110 [cs.CL]. Disponível em: [link].
Devlin, J. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
Touvron, H. et al. LLaMA: Open and Efficient Foundation Language Models. 2023. arXiv: 2302.13971 [cs.CL]. Disponível em: [link].
Socher, R. Introducing a Conditional Transformer Language Model for Controllable Generation. 2019. Disponível em: [link]. Acesso em: 11 set. 2019.
Conover, M. et al. Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM. 2023. Disponível em: [link]. Acesso em: 12 abr. 2023.
Vassilev, A. et al. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. Apostol Vassilev, Alina Oprea, Alie Fordyce, Hyrum Andersen, 2024.
Stanton, B.; Jensen, T. Trust and Artificial Intelligence. en. NIST Interagency/Internal Report (NISTIR), National Institute of Standards e Technology, Gaithersburg, MD, mar. 2021. Disponível em: [link].
European Union Agency for Cybersecurity (ENISA). Securing Machine Learning Algorithms. 2021. [link]. ISBN: 978-92-9204-543-2, DOI: 10.2824/874249, Catalogue Nr.: TP-06-21-153-EN-N.
MITRE. MITRE ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems). Visitado em Dezembro de 2023. 2023. Disponível em: [link].
Xu, J. et al. Bot-Adversarial Dialogue for Safe Conversational Agents. In: Toutanova, K. et al. (Ed.). Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, jun. 2021. P. 2950–2968. DOI: 10.18653/v1/2021.naacl-main.235.
Li, Y.; Liu, Q. A comprehensive review study of cyber-attacks and cyber security; Emerging trends and recent developments. Energy Reports, v. 7, p. 8176–8186, 2021. ISSN 2352-4847. DOI: 10.1016/j.egyr.2021.08.126.
Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems, v. 30, 2017.
Yao, Y. et al. A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly. High-Confidence Computing, v. 4, n. 2, p. 100211, 2024. ISSN 2667-2952. DOI: 10.1016/j.hcc.2024.100211.
Biggio, B.; Nelson, B.; Laskov, P. Poisoning Attacks against Support Vector Machines. 2013. arXiv: 1206.6389 [cs.LG]. Disponível em: [link].
Kurita, K.; Michel, P.; Neubig, G.Weight Poisoning Attacks on Pre-trained Models. 2020. Visitado em Outubro de 2024. Disponível em: [link].
Ge, S. et al. MART: Improving LLM Safety with Multi-round Automatic Red-Teaming. 2023. arXiv: 2311.07689 [cs.CL]. Disponível em: [link].
Wan, A. et al. Poisoning Language Models During Instruction Tuning. 2023. Visitado em Outubro de 2024. Disponível em: [link].
Zhang, Y. et al. Siren’s song in the AI ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219, 2023.
Ding, X. et al. HPC-GPT: Integrating Large Language Model for High-Performance Computing. In: PROCEEDINGS of the SC ’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis. Denver, CO, USA: Association for Computing Machinery, 2023. (SC-W ’23), p. 951–960. ISBN 9798400707858. DOI: 10.1145/3624062.3624172.
Brown, T. B. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.
Liang, P. et al. Holistic Evaluation of Language Models. 2023. arXiv: 2211. 09110 [cs.CL]. Disponível em: [link].
Devlin, J. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
Touvron, H. et al. LLaMA: Open and Efficient Foundation Language Models. 2023. arXiv: 2302.13971 [cs.CL]. Disponível em: [link].
Socher, R. Introducing a Conditional Transformer Language Model for Controllable Generation. 2019. Disponível em: [link]. Acesso em: 11 set. 2019.
Conover, M. et al. Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM. 2023. Disponível em: [link]. Acesso em: 12 abr. 2023.
Publicado
27/11/2024
Como Citar
OLIVEIRA, Diego E. G. C. de; MIERS, Charles C..
LLM vs. LLM: uso de red team e AML para avaliar a segurança de LLMs. In: ESCOLA REGIONAL DE REDES DE COMPUTADORES (ERRC), 21. , 2024, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 192-195.
DOI: https://doi.org/10.5753/errc.2024.4670.
