Weightless Neural Trees
Resumo
This work introduces Weightless Neural Tree (WNT), a neurosymbolic model based on the fusion of Decision Tree (DT) and Weightless Neural Network (WNN). A new learning algorithm is proposed to extract rules learned by a DT and design flexible WNN architectures based on tree discriminator generation and binary conditional mapping. Single-depth, boundary, and neighborhood mappings are also proposed to optimize the capture of interpartition patterns. Finally, the WNT model is extended in the Weightless Neural Forest (WNF) with the addition of two ensemble methods that utilize rules learned by a Random Forest (RF). Experiments on ten classification datasets evaluate accuracy, F1-score, and model size under different hyperparameter configurations. WNT substantially reduces model size relative to the evaluated DT and WiSARD baselines, with reductions of up to 96%, while achieving competitive accuracy on most datasets. WNF further improves the accuracy of WNT and achieves up to 96.25% lower model size than the evaluated Random Forest baseline, while remaining competitive with Random Forest and XGBoost on the evaluated datasets. These results indicate that tree-guided architecture design is a promising direction for memory-efficient weightless neural models. https://github.com/leandro-santiago/wnt
Referências
Bacellar, A., Susskind, Z., Villon, L., Miranda, I., Santiago, L., Dutra, D., Breternitz, M., JOHN, L., Lima, P., and França, F. (2022). Distributive thermometer: A new unary encoding for weightless neural networks. pages 31–36.
Bacellar, A. T. L., Susskind, Z., Breternitz Jr., M., John, E., John, L. K., Lima, P. M. V., and França, F. M. G. (2024). Differentiable weightless neural networks. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
Isonuma, M., Mori, J., Bollegala, D., and Sakata, I. (2020). Tree-Structured Neural Topic Model. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 800–806, Online. Association for Computational Linguistics.
Li, H., Song, J., Xue, M., Zhang, H., and Song, M. (2025). A survey of neural trees: Co-evolving neural networks and decision trees. IEEE Transactions on Neural Networks and Learning Systems, 36(7):11718–11737.
Ludermir, T. B., Carvalho, A. C. P. d. L. F. d., Braga, A. P., and De Souto, M. C. (1999). Weightless neural models: a review of current and past works. Neural Computing Surveys, 2:41–61.
Rana, K. K. et al. (2014). A survey on decision tree algorithm for classification. International journal of Engineering development and research, 2(1):1–5.
Santiago, L., Verona, L., Rangel, F., Firmino, F., Menasché, D. S., Caarls, W., Breternitz Jr, M., Kundu, S., Lima, P. M., and França, F. M. (2020). Weightless neural networks as memory segmented bloom filters. Neurocomputing, 416:292–304.
Susskind, Z., Arora, A., Miranda, I. D. S., Bacellar, A. T. L., Villon, L. A. Q., Katopodis, R. F., de Araújo, L. S., Dutra, D. L. C., Lima, P. M. V., França, F. M. G., Breternitz Jr., M., and John, L. K. (2023). Uleen: A novel architecture for ultra-low-energy edge neural networks. ACM Trans. Archit. Code Optim., 20(4).
Tanno, R., Arulkumaran, K., Alexander, D., Criminisi, A., and Nori, A. (2019). Adaptive neural trees. In Chaudhuri, K. and Salakhutdinov, R., editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6166–6175. PMLR.
Unwin, A. and Kleinman, K. (2021). The iris data set: In search of the source of virginica. Significance, 18.
