Skip to main content
SpringerLink
Log in

When External Knowledge Does Not Aggregate in Named Entity Recognition

  • Conference paper
  • First Online:
Intelligent Systems (BRACIS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13074))

Included in the following conference series:

  • 959 Accesses

Abstract

In the different areas of knowledge, textual data are important sources of information. This way, Information Extraction methods have been developed to identify and structure information present in textual documents. In particular there is the Named Entity Recognition (NER) task, which consists of using methods to identify Named Entities, such as Person, Place, among others, in texts, using techniques from Natural Language Processing and Machine Learning. Recent works explored the use of external sources of knowledge to boost the Machine Learning models with sets of domain specific relevant information for the NER task. This work aims to evaluate the aggregation of external knowledge, in the form of Gazetter and Knowledge Graphs, for NER task. Our approach is composed of two steps: i) generation of embeddings, ii) definition and training of the Machine Learning methods. The experiments were conducted on four English datasets, and their results show that the applied strategies for external knowledge integration did not bring great gains to the models, as expressed by F1-Score metric. In the performed experiments, there was an F1-score increase in 17 of the 32 cases where external knowledge was used, but in most cases the gains were lesser than 0.5% in F1-score. In some scenarios the aggregated external knowledge does not capture relevant content, thus not being necessarily beneficial to the methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://developers.google.com/freebase/.

References

  1. Akbik, A., Blythe, D., Vollgraf, R.: Contextual string embeddings for sequence labeling. In: Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA, pp. 1638–1649. Association for Computational Linguistics (August 2018). https://www.aclweb.org/anthology/C18-1139

  2. Baevski, A., Edunov, S., Liu, Y., Zettlemoyer, L., Auli, M.: Cloze-driven pretraining of self-attention networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 5360–5369. Association for Computational Linguistics (November 2019). https://doi.org/10.18653/v1/D19-1539. https://www.aclweb.org/anthology/D19-1539

  3. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Computat. Linguist. 5, 135–146 (2017). https://doi.org/10.1162/tacl_a_00051. https://www.aclweb.org/anthology/Q17-1010

  4. Bordes, A., Usunier, N., Garcia-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS 2013, vol. 2, pp. 2787–2795. Curran Associates Inc., Red Hook (2013). https://doi.org/10.5555/2999792.2999923

  5. Chiu, J.P., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016). https://www.aclweb.org/anthology/Q16-1026

  6. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011). https://doi.org/10.5555/1953048.2078186

  7. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (June 2019). https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423

  8. Ding, R., Xie, P., Zhang, X., Lu, W., Li, L., Si, L.: A neural multi-digraph model for Chinese NER with gazetteers. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 1462–1467. Association for Computational Linguistics (July 2019). https://doi.org/10.18653/v1/P19-1141. https://aclanthology.org/P19-1141

  9. Freire, N., Borbinha, J., Calado, P.: An approach for named entity recognition in poorly structured data. In: Simperl, E., Cimiano, P., Polleres, A., Corcho, O., Presutti, V. (eds.) ESWC 2012. LNCS, vol. 7295, pp. 718–732. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30284-8_55

    Chapter  Google Scholar 

  10. Gorinski, P.J., et al.: Named entity recognition for electronic health records: a comparison of rule-based and machine learning approaches. CoRR abs/1903.03985. arXiv:1903.03985 (2019)

  11. Goyal, A., Gupta, V., Kumar, M.: Recent named entity recognition and classification techniques: a systematic review. Comput. Sci. Rev. 29, 21–43 (2018). https://doi.org/10.1016/j.cosrev.2018.06.001

  12. Habibi, M., Weber, L., Neves, M., Wiegandt, D.L., Leser, U.: Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33(14), i37–i48 (2017). https://doi.org/10.1093/bioinformatics/btx228

  13. Han, X., et al.: OpenKE: an open toolkit for knowledge embedding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Brussels, Belgium, pp. 139–144. Association for Computational Linguistics (November 2018). https://doi.org/10.18653/v1/D18-2024. https://www.aclweb.org/anthology/D18-2024

  14. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, California, pp. 260–270. Association for Computational Linguistics (June 2016). https://doi.org/10.18653/v1/N16-1030. https://www.aclweb.org/anthology/N16-1030

  15. Lange, D., Böhm, C., Naumann, F.: Extracting structured information from Wikipedia articles to populate infoboxes. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM 2010, pp. 1661–1664. Association for Computing Machinery, New York (2010). https://doi.org/10.1145/1871437.1871698

  16. Li, X., Sun, X., Meng, Y., Liang, J., Wu, F., Li, J.: Dice loss for data-imbalanced NLP tasks. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 465–476. Association for Computational Linguistics (July 2020). https://doi.org/10.18653/v1/2020.acl-main.45. https://www.aclweb.org/anthology/2020.acl-main.45

  17. Lin, H., Lu, Y., Han, X., Sun, L., Dong, B., Jiang, S.: Gazetteer-enhanced attentive neural networks for named entity recognition. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 6232–6237. Association for Computational Linguistics (November 2019). https://doi.org/10.18653/v1/D19-1646. https://aclanthology.org/D19-1646

  18. Liu, J., Pasupat, P., Cyphers, S., Glass, J.: Asgard: a portable architecture for multilingual dialogue systems. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8386–8390 (2013). https://doi.org/10.1109/ICASSP.2013.6639301

  19. Liu, K., El-Gohary, N.: Ontology-based semi-supervised conditional random fields for automated information extraction from bridge inspection reports. Autom. Constr. 81 (2017). https://doi.org/10.1016/j.autcon.2017.02.003

  20. Liu, T., Yao, J.G., Lin, C.Y.: Towards improving neural named entity recognition with gazetteers. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 5301–5307. Association for Computational Linguistics (July 2019). https://doi.org/10.18653/v1/P19-1524. https://www.aclweb.org/anthology/P19-1524

  21. Mahdisoltani, F., Biega, J., Suchanek, F.M.: Yago3: a knowledge base from multilingual Wikipedias (2013). https://hal-imt.archives-ouvertes.fr/hal-01699874/

  22. Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press (2008). https://doi.org/10.1017/CBO9780511809071

  23. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Bengio, Y., LeCun, Y. (eds.) 1st International Conference on Learning Representations, Workshop Track Proceedings, ICLR 2013, Scottsdale, Arizona, USA, 2–4 May 2013 (2013). arXiv:1301.3781

  24. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. CoRR abs/1310.4546. arXiv:1310.4546 (2013)

  25. Nadeau, D., Sekine, S.: A survey of named entity recognition and classification. Lingvisticæ Investigationes 30(1), 3–26 (2007). https://doi.org/10.1075/li.30.1.03nad. https://www.jbe-platform.com/content/journals/10.1075/li.30.1.03nad

  26. Nurdin, A., Maulidevi, N.U.: 5w1h information extraction with CNN-bidirectional LSTM. J. Phys. Conf. Ser. 978, 012078 (2018). https://doi.org/10.1088/1742-6596/978/1/012078

  27. Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1532–1543. Association for Computational Linguistics (October 2014). https://doi.org/10.3115/v1/D14-1162. https://www.aclweb.org/anthology/D14-1162

  28. Rais, M., Lachkar, A., Lachkar, A., Ouatik, S.E.A.: A comparative study of biomedical named entity recognition methods based machine learning approach. In: 2014 3rd IEEE International Colloquium in Information Science and Technology (CIST), pp. 329–334 (October 2014). https://doi.org/10.1109/CIST.2014.7016641

  29. Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: Proceedings of the 13th Conference on Computational Natural Language Learning, CoNLL 2009, USA, pp. 147–155. Association for Computational Linguistics (2009)

    Google Scholar 

  30. Seyler, D., Dembelova, T., Del Corro, L., Hoffart, J., Weikum, G.: A study of the importance of external knowledge in the named entity recognition task. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Melbourne, Australia, pp. 241–246. Association for Computational Linguistics (July 2018). https://doi.org/10.18653/v1/P18-2039. https://aclanthology.org/P18-2039

  31. Tjong Kim Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In: Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL 2003, CoNLL 2003, USA, vol. 4, pp. 142–147. Association for Computational Linguistics (2003). https://doi.org/10.3115/1119176.1119195

  32. Xiaofeng, M., Wei, W., Aiping, X.: Incorporating token-level dictionary feature into neural model for named entity recognition. Neurocomputing 375, 43–50 (2020). https://doi.org/10.1016/j.neucom.2019.09.005. https://www.sciencedirect.com/science/article/pii/S0925231219312652

  33. Yadav, V., Bethard, S.: A survey on recent advances in named entity recognition from deep learning models. In: Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA, pp. 2145–2158. Association for Computational Linguistics (August 2018). https://www.aclweb.org/anthology/C18-1182

Download references

Author information

Authors and Affiliations

  1. Institute of Geosciences and Exact Sciences, UNESP - São Paulo State University, Rio Claro, SP, Brazil

    Pedro Ivo Monteiro Privatto & Ivan Rizzo Guilherme

Authors
  1. Pedro Ivo Monteiro Privatto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ivan Rizzo Guilherme
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Universidade Federal de Sergipe, São Cristóvão, Brazil

    André Britto

  2. Universidade de São Paulo, São Paulo, Brazil

    Karina Valdivia Delgado

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Privatto, P.I.M., Guilherme, I.R. (2021). When External Knowledge Does Not Aggregate in Named Entity Recognition. In: Britto, A., Valdivia Delgado, K. (eds) Intelligent Systems. BRACIS 2021. Lecture Notes in Computer Science(), vol 13074. Springer, Cham. https://doi.org/10.1007/978-3-030-91699-2_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91699-2_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91698-5

  • Online ISBN: 978-3-030-91699-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation