Abstract
Semi-supervised learning has received attention from researchers, as it allows one to exploit the structure of unlabeled data to achieve competitive classification results with much fewer labels than supervised approaches. The Local and Global Consistency (LGC) algorithm is one of the most well-known graph-based semi-supervised (GSSL) classifiers. Notably, its solution can be written as a linear combination of the known labels. The coefficients of this linear combination depend on a parameter \(\alpha \), determining the decay of the reward over time when reaching labeled vertices in a random walk. In this work, we discuss how removing the self-influence of a labeled instance may be beneficial, and how it relates to leave-one-out error. Moreover, we propose to minimize this leave-one-out loss with automatic differentiation. Within this framework, we propose methods to estimate label reliability and diffusion rate. Optimizing the diffusion rate is more efficiently accomplished with a spectral representation. Results show that the label reliability approach competes with robust \(\ell _1\)-norm methods and that removing diagonal entries reduces the risk of overfitting and leads to suitable criteria for parameter selection.
This study was financed in part by the Coordenação de Aperfeiçoamento de Nível Superior - Brasil (CAPES) - Finance Code 001, and São Paulo Research Foundation (FAPESP) grant #18/01722-3.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). http://tensorflow.org/, software available from tensorflow.org
de Aquino Afonso, B.K.: Analysis of Label Noise in Graph-Based Semi-supervised Learning. Master’s thesis (2020)
Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-supervised Learning. MIT Press, Cambridge (2006). http://www.kyb.tuebingen.mpg.de/ssl-book
de Aquino Afonso, B.K., Berton, L.: Identifying noisy labels with a transductive semi-supervised leave-one-out filter. Pattern Recognit. Lett. 140, 127–134 (2020). https://doi.org/10.1016/j.patrec.2020.09.024. http://www.sciencedirect.com/science/article/pii/S0167865520303603
Fergus, R., Weiss, Y., Torralba, A.: Semi-supervised learning in gigantic image collections. In: Advances in Neural Information Processing Systems, pp. 522–530 (2009)
Gong, C., Zhang, H., Yang, J., Tao, D.: Learning with inadequate and incorrect supervision. In: 2017 IEEE International Conference on Data Mining (ICDM), pp. 889–894. IEEE (2017)
Johnson, J., Douze, M., Jégou, H.: Billion-scale similarity search with GPUs. IEEE Trans. Big Data (2019)
Kearnes, S., McCloskey, K., Berndl, M., Pande, V., Riley, P.: Molecular graph convolutions: moving beyond fingerprints. J. Comput.-Aided Mol. Des. 30(8), 595–608 (2016). https://doi.org/10.1007/s10822-016-9938-8
Krijthe, J.H.: Robust semi-supervised learning: projections, limits and constraints. Ph.D. thesis, Leiden University (2018)
Lu, Z., Gao, X., Wang, L., Wen, J.R., Huang, S.: Noise-robust semi-supervised learning by large-scale sparse coding. In: AAAI, pp. 2828–2834 (2015)
Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)
Miyato, T., Maeda, S.I., Ishii, S., Koyama, M.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1979–1993 (2018)
Shao, Y., Sang, N., Gao, C., Ma, L.: Probabilistic class structure regularized sparse representation graph for semi-supervised hyperspectral image classification. Pattern Recognit. 63, 102–114 (2017)
Van Engelen, J.E., Hoos, H.H.: A survey on semi-supervised learning. Mach. Learn. 109(2), 373–440 (2020). https://doi.org/10.1007/s10994-019-05855-6
Wang, Y.X., Sharpnack, J., Smola, A.J., Tibshirani, R.J.: Trend filtering on graphs. J. Mach. Learn. Res. 17(1), 3651–3691 (2016)
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W.L., Leskovec, J.: Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 974–983 (2018)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems, pp. 321–328 (2004)
Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the Twentieth International Conference on International Conference on Machine Learning, pp. 912–919. AAAI Press (2003)
Catunda, J.P.K., da Silva, A.T., Berton, L.: Car plate character recognition via semi-supervised learning. In: 2019 8th Brazilian Conference on Intelligent Systems (BRACIS), pp. 735–740. IEEE (2019)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
de Aquino Afonso, B.K., Berton, L. (2021). Optimizing Diffusion Rate and Label Reliability in a Graph-Based Semi-supervised Classifier. In: Britto, A., Valdivia Delgado, K. (eds) Intelligent Systems. BRACIS 2021. Lecture Notes in Computer Science(), vol 13073. Springer, Cham. https://doi.org/10.1007/978-3-030-91702-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-91702-9_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91701-2
Online ISBN: 978-3-030-91702-9
eBook Packages: Computer ScienceComputer Science (R0)