Detecção de Depressão nas Mídias Sociais usando Transformers com Aprendizado Federado
Resumo
Diversas iniciativas investigam a detecção automática de depressão através de publicações em redes sociais utilizando de modelos de aprendizado de máquina. A forma mais adequada de obter conjuntos de dados seria pelo consentimento dos usuários em compartilhar seus dados. Porém, compartilhar estas publicações para outros grupos de pesquisa nem sempre é recomendado dada a sensibilidade dos dados. Uma solução para este problema, é se valer de técnicas de Aprendizado Federado para trocar modelos no lugar de dados. Neste trabalho, investigamos como combinar Transformers, a arquitetura estado da arte para word embedding, treinados para abordar a detecção automática de depressão nas mídias sociais.
Palavras-chave:
Depressão, Aprendizado federado, Aprendizado de máquina, BERT
Referências
Beck et al. (1961). An Inventory for Measuring Depression. Archives of General Psychiatry, 4(6):561–571.
Beutel et al. (2020). Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390.
Devlin et al. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
McMahan et al. (2017). Communication-Efficient Learning of Deep Networks from Decentralized Data. In Singh, A. and Zhu, J., editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 1273–1282. PMLR.
Parapar et al. (2021). Overview of erisk 2021: Early risk prediction on the internet. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 12th International Conference of the CLEF Association, CLEF 2021, Virtual Event, September 21–24, 2021, Proceedings, page 324–344, Berlin, Heidelberg. Springer-Verlag.
Vaswani et al. (2017). Attention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Wolf T. et al. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
World Health Organization (2017). Technical report, Depression and Other Common Mental Disorders: Global Health Estimates. Geneva: World Health Organization; 2017. Licence: CC BY-NC-SA 3.0 IGO
Beutel et al. (2020). Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390.
Devlin et al. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
McMahan et al. (2017). Communication-Efficient Learning of Deep Networks from Decentralized Data. In Singh, A. and Zhu, J., editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 1273–1282. PMLR.
Parapar et al. (2021). Overview of erisk 2021: Early risk prediction on the internet. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 12th International Conference of the CLEF Association, CLEF 2021, Virtual Event, September 21–24, 2021, Proceedings, page 324–344, Berlin, Heidelberg. Springer-Verlag.
Vaswani et al. (2017). Attention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Wolf T. et al. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
World Health Organization (2017). Technical report, Depression and Other Common Mental Disorders: Global Health Estimates. Geneva: World Health Organization; 2017. Licence: CC BY-NC-SA 3.0 IGO
Publicado
21/06/2023
Como Citar
VASCONCELOS, Arthur B.; BRUM, Rafaela; PAES, Aline; DRUMMOND, Lúcia Maria de Assumpção.
Detecção de Depressão nas Mídias Sociais usando Transformers com Aprendizado Federado. In: ESCOLA REGIONAL DE ALTO DESEMPENHO DO RIO DE JANEIRO (ERAD-RJ), 8. , 2023, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 11-13.
DOI: https://doi.org/10.5753/eradrj.2023.231537.
