Simplifying Simplex Convolutional Networks
Resumo
Topological deep learning (TDL) has recently emerged as a family of neural networks for data on topological domains (e.g., graphs and cellular complexes). In the context of graph data, TDL provides a recipe for leveraging high-order information, e.g., from cliques, to boost the expressiveness of graph neural networks. This additional power comes at a computational cost, stemming from message passing between higher-order structures. In this paper, we alleviate the computing toll of TDL by proposing SimpleXCN, a linear convolutional model for simplicial complexes. SimpleXCN derives naturally from removing non-linearities of base TDL architectures. Despite its simplicity, SimpleXCN remains competitive with established topological neural networks in node classification tasks. Our experiments show a reduction of approximately 25% in time and 10 times in the number of parameters, and consequently a significant reduction in memory usage of up to 50% compared to other SNNs.
Palavras-chave:
Graph Neural Networks, Node Classification, Topological Deep Learning
Referências
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: A next-generation hyperparameter optimization framework. In The 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 2623–2631, 2019.
Bodnar, C., Frasca, F., Wang, Y., Otter, N., Montufar, G. F., Lió, P., and Bronstein, M. Weisfeiler and Lehman go topological: Message passing simplicial networks. In Proceedings of the 38th International Conference on Machine Learning, M. Meila and T. Zhang (Eds.). Proceedings of Machine Learning Research, vol. 139. PMLR, pp. 1026–1037, 2021.
Bojchevski, A. and Günnemann, S. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In International Conference on Learning Representations, 2018.
Chen, J., Wang, R., Wang, M., and Wei, G.-W. Mutations strengthened sars-cov-2 infectivity. Journal of Molecular Biology 432 (19): 5212–5226, 2020.
Hajij, M., Zamzmi, G., Papamarkou, T., Miolane, N., Guzmán-Sáenz, A., and Ramamurthy, K. N. Higher-order attention networks. arXiv preprint arXiv:2206.00606 2 (3): 4, 2022.
Horn, M., De Brouwer, E., Moor, M., Moreau, Y., Rieck, B., and Borgwardt, K. Topological graph neural networks. In International Conference on Representation Learning (ICLR), 2022.
Kahle, M. The neighborhood complex of a random graph. Journal of Combinatorial Theory, Series A 114 (2): 380–387, 2007.
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Nguyen, D. D., Cang, Z., Wu, K., Wang, M., Cao, Y., and Wei, G.-W. Mathematical deep learning for pose and binding affinity prediction and ranking in d3r grand challenges. Journal of Computer-Aided Molecular Design 33 (1): 71–82, 2019.
Papamarkou, T., Birdal, T., Bronstein, M., Carlsson, G., Curry, J., Gao, Y., Hajij, M., Kwitt, R., Liò, P., Lorenzo, P. D., Maroulas, V., Miolane, N., Nasrin, F., Ramamurthy, K. N., Rieck, B., Scardapane, S., Schaub, M. T., Veličković, P., Wang, B., Wang, Y., Wei, G.-W., and Zamzmi, G. Position: Topological deep learning is the new frontier for relational learning. In International Conference on Machine Learning (ICML), 2024.
Papillon, M., Sanborn, S., Hajij, M., and Miolane, N. Architectures of topological deep learning: A survey on topological neural networks. ArXiv e-prints, 2023.
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look at evaluation of gnns under heterophily: Are we really making progress? In The Eleventh International Conference on Learning Representations, 2023.
PyT-Team. Topobenchmarkx, 2024.
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. The graph neural network model. IEEE Transactions on Neural Networks 20 (1): 61–80, 2009.
Weisfeiler, B. and Leman, A. The reduction of a graph to canonical form and the algebra which appears therein. nti, Series 2 (9): 12–16, 1968.
Wilcoxon, F. Individual comparisons by ranking methods. Biometrics Bulletin 1 (6): 80–83, 1945.
Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., and Weinberger, K. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning, K. Chaudhuri and R. Salakhutdinov (Eds.). Proceedings of Machine Learning Research, vol. 97. PMLR, pp. 6861–6871, 2019.
Yang, R., Sala, F., and Bogdan, P. Efficient representation learning for higher-order data with simplicial complexes. In Proceedings of the First Learning on Graphs Conference, B. Rieck and R. Pascanu (Eds.). Proceedings of Machine Learning Research, vol. 198. PMLR, pp. 13:1–13:21, 2022.
Yang, Z., Cohen, W., and Salakhudinov, R. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning. PMLR, pp. 40–48, 2016.
Bodnar, C., Frasca, F., Wang, Y., Otter, N., Montufar, G. F., Lió, P., and Bronstein, M. Weisfeiler and Lehman go topological: Message passing simplicial networks. In Proceedings of the 38th International Conference on Machine Learning, M. Meila and T. Zhang (Eds.). Proceedings of Machine Learning Research, vol. 139. PMLR, pp. 1026–1037, 2021.
Bojchevski, A. and Günnemann, S. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In International Conference on Learning Representations, 2018.
Chen, J., Wang, R., Wang, M., and Wei, G.-W. Mutations strengthened sars-cov-2 infectivity. Journal of Molecular Biology 432 (19): 5212–5226, 2020.
Hajij, M., Zamzmi, G., Papamarkou, T., Miolane, N., Guzmán-Sáenz, A., and Ramamurthy, K. N. Higher-order attention networks. arXiv preprint arXiv:2206.00606 2 (3): 4, 2022.
Horn, M., De Brouwer, E., Moor, M., Moreau, Y., Rieck, B., and Borgwardt, K. Topological graph neural networks. In International Conference on Representation Learning (ICLR), 2022.
Kahle, M. The neighborhood complex of a random graph. Journal of Combinatorial Theory, Series A 114 (2): 380–387, 2007.
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Nguyen, D. D., Cang, Z., Wu, K., Wang, M., Cao, Y., and Wei, G.-W. Mathematical deep learning for pose and binding affinity prediction and ranking in d3r grand challenges. Journal of Computer-Aided Molecular Design 33 (1): 71–82, 2019.
Papamarkou, T., Birdal, T., Bronstein, M., Carlsson, G., Curry, J., Gao, Y., Hajij, M., Kwitt, R., Liò, P., Lorenzo, P. D., Maroulas, V., Miolane, N., Nasrin, F., Ramamurthy, K. N., Rieck, B., Scardapane, S., Schaub, M. T., Veličković, P., Wang, B., Wang, Y., Wei, G.-W., and Zamzmi, G. Position: Topological deep learning is the new frontier for relational learning. In International Conference on Machine Learning (ICML), 2024.
Papillon, M., Sanborn, S., Hajij, M., and Miolane, N. Architectures of topological deep learning: A survey on topological neural networks. ArXiv e-prints, 2023.
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look at evaluation of gnns under heterophily: Are we really making progress? In The Eleventh International Conference on Learning Representations, 2023.
PyT-Team. Topobenchmarkx, 2024.
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. The graph neural network model. IEEE Transactions on Neural Networks 20 (1): 61–80, 2009.
Weisfeiler, B. and Leman, A. The reduction of a graph to canonical form and the algebra which appears therein. nti, Series 2 (9): 12–16, 1968.
Wilcoxon, F. Individual comparisons by ranking methods. Biometrics Bulletin 1 (6): 80–83, 1945.
Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., and Weinberger, K. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning, K. Chaudhuri and R. Salakhutdinov (Eds.). Proceedings of Machine Learning Research, vol. 97. PMLR, pp. 6861–6871, 2019.
Yang, R., Sala, F., and Bogdan, P. Efficient representation learning for higher-order data with simplicial complexes. In Proceedings of the First Learning on Graphs Conference, B. Rieck and R. Pascanu (Eds.). Proceedings of Machine Learning Research, vol. 198. PMLR, pp. 13:1–13:21, 2022.
Yang, Z., Cohen, W., and Salakhudinov, R. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning. PMLR, pp. 40–48, 2016.
Publicado
17/11/2024
Como Citar
DUARTE, Gabriel; ARRUDA, Tamara; MESQUITA, Diego; SOUZA, Amauri H..
Simplifying Simplex Convolutional Networks. In: SYMPOSIUM ON KNOWLEDGE DISCOVERY, MINING AND LEARNING (KDMILE), 12. , 2024, Belém/PA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 137-144.
ISSN 2763-8944.
DOI: https://doi.org/10.5753/kdmile.2024.244748.
