KalWeNN: Kalman-Filtered Memory for Weightless Neural Network Classifiers

• Helena A. B. Moreno Universidade Federal do Rio de Janeiro (UFRJ) https://orcid.org/0009-0007-9893-0749
• Natasha C. da Fonseca Universidade Federal do Rio de Janeiro (UFRJ) https://orcid.org/0009-0006-0481-8266
• Gustavo R. Villela Universidade Federal do Rio de Janeiro (UFRJ) https://orcid.org/0009-0002-0223-6992
• Priscila M. V. Lima Universidade Federal do Rio de Janeiro (UFRJ) https://orcid.org/0000-0002-8515-9904
• Pablo Rangel Universidade Federal do Rio de Janeiro (UFRJ)
• Leandro S. de Araújo Universidade Federal Fluminense (UFF) https://orcid.org/0000-0002-3631-8761
• Claudio M. de Farias Universidade Federal do Rio de Janeiro (UFRJ)

DOI: https://doi.org/10.5753/sbbd_estendido.2026.249740

Resumo

Weightless Neural Networks (WNNs) such as WiSARD learn by storing information directly in RAM-based memory structures, enabling fast training and efficient hardware implementation. However, conventional WiSARD memories store only binary activations or occurrence counts, providing no measure of confidence in the stored information. This paper introduces KalWeNN (Kalman Weightless Neural Network), a weightless classifier that replaces the traditional counting memory with Kalman filter-based memory cells. Each memory address maintains both an estimate of its association with a class and the uncertainty of that estimate. During training, address activations are treated as noisy observations and incorporated through recursive Kalman update. We evaluate KalWeNN on twelve classification datasets under two architectural configurations and compare it with a standard WiSARD implementation. Results show that KalWeNN consistently matches or improves classification accuracy, achieving approximately 93% accuracy on MNIST. More importantly, it substantially improves run-to-run stability, often reducing performance variability by a factor of four to five. These gains are obtained with only modest increases in memory usage and execution time, preserving the lightweight nature of weightless neural networks.

Palavras-chave: Weightless Neural Network, Kalman Filter, Classification Task, Machine Learning, KalWeNN

Referências

Aleksander, I., Thomas, W. V., and Bowden, P. A. (1984). WISARD: a radical step forward in image recognition. Sensor Review, 4(3):120–124.

Hammond, J. E., Soderstrom, T. A., Korgel, B. A., and Baldea, M. (2025). A selective Kalman filtering approach to online neural network updating under system drift. Scientific Reports, 15(1):43577.

Haykin, S., editor (2001). Kalman Filtering and Neural Networks. John Wiley & Sons, New York, NY, USA.

Khodarahmi, M. and Maihami, V. (2023). A review on kalman filter models. Archives of Computational Methods in Engineering, 30(1):727–747.

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature, 521(7553):436–444.

Santiago, L., Verona, L., Rangel, F., Firmino, F., Menasché, D. S., Caarls, W., Breternitz Jr, M., Kundu, S., Lima, P. M., and França, F. M. (2020). Weightless neural networks as memory segmented bloom filters. Neurocomputing, 416:292–304.

Susskind, Z., Arora, A., Miranda, I. D. S., Villon, L. A. Q., Katopodis, R. F., Araújo, L. S. d., Dutra, D. L. C., Lima, P. M. V., França, F. M. G., Breternitz Jr., M., and John, L. K. (2022). Weightless neural networks for efficient edge inference. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT), pages 279–290, Chicago, IL, USA.