Autoencoders Assimétricos para a Compressão de Dados IoT

• Mateus da Silva Gilbert UFRJ
• Marcello Luiz R. de Campos UFRJ
• Miguel Elias M. Campista UFRJ

DOI: https://doi.org/10.5753/sbrc.2023.521

Abstract

IoT devices typically have severe limitations regarding energy consumption and the number of local computations. Thus, finding solutions that reduce these two issues are always welcome. The generated data may have intrinsic redundancies that allow its compression without loss of information, reducing the amount of data transmitted over the network, one of the most energy-consuming tasks for IoT devices. Consequently, many solutions using neural networks have emerged to reduce data transmission in IoT networks. This paper follows this trend to propose Asymmetric Autoencoders (AAEs), which have fewer neural network layers at the encoder than at the decoder. The proposed structure modifies typical autoencoders with the same number of layers at both the encoder and the decoder. The key idea of the asymmetrical design is to minimize the number of parameters stored and computations performed in IoT devices. Our experiments show improvements compared with symmetrical autoencoders, achieving lower reconstruction errors using temporal samples from a single sensor.

References

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org.

Akyildiz, I. F., Su, W., Sankarasubramaniam, Y., and Cayirci, E. (2002). Wireless sensor networks: a survey. Computer networks, 38(4):393-422.

Alsheikh, M. A., Lin, S., Niyato, D., and Tan, H.-P. (2016). Rate-distortion balanced data compression for wireless sensor networks. IEEE Sensors Journal, 16(12):5072-5083.

Bales, R., Cui, G., Rice, R., Meng, X., Zhang, Z., Hartsough, P., Glaser, S., and Conklin, M. (2020). Snow depth, air temperature, humidity, soil moisture and temperature, and solar radiation data from the basin-scale wireless-sensor network in American River Hydrologic Observatory (ARHO). https://doi.org/10.6071/M39Q2V. Accessed in 08/06/2020.

Bochie, K., Gilbert, M. S., Gantert, L., Barbosa, M. S., Medeiros, D. S., and Campista, M. E. M. (2021). A survey on deep learning for challenged networks: Applications and trends. Journal of Network and Computer Applications, page 103213.

Charte, D., Charte, F., García, S., del Jesus, M. J., and Herrera, F. (2018). A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines. Information Fusion, 44:78-96.

Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems, 2(4):303-314.

Dozat, T. (2016). Incorporating nesterov momentum into adam. In International Conference on Learning Representations (ICLR).

Dumoulin, V. and Visin, F. (2016). A guide to convolution arithmetic for deep learning. arXiv preprint arXiv:1603.07285.

Ghosh, A. M. and Grolinger, K. (2019). Deep learning: Edge-cloud data analytics for IoT. In 2019 IEEE Canadian Conference of Electrical and Computer Engineering (CCECE), pages 1-7. IEEE.

Gilbert, M. d. S. (2021). Redução de dados em redes de sensores utilizando redes neurais. Projeto Final de Graduação. [link].

Gubbi, J., Buyya, R., Marusic, S., and Palaniswami, M. (2013). Internet of things (IoT): A vision, architectural elements, and future directions. Future generation computer systems, 29(7):1645-1660.

Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., del Río, J. F., Wiebe, M., Peterson, P., Gérard-Marchant, P., Sheppard, K., Reddy, T., Weckesser, W., Abbasi, H., Gohlke, C., and Oliphant, T. E. (2020). Array programming with NumPy. Nature, 585(7825):357-362.

Iwana, B. K. and Uchida, S. (2021). An empirical survey of data augmentation for time series classification with neural networks. Plos one, 16(7):e0254841.

Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S. (2017). Self-normalizing neural networks. Advances in neural information processing systems, 30:971-980.

LeCun, Y., Bengio, Y., et al. (1995). Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10):1995.

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436-444.

Mohammadi, M., Al-Fuqaha, A., Sorour, S., and Guizani, M. (2018). Deep learning for iot big data and streaming analytics: A survey. IEEE Communications Surveys & Tutorials, 20(4):2923-2960.

Nesterov, Y. E. (1983). A method for solving the convex programming problem with convergence rate o (1/k^ 2). In Dokl. akad. nauk Sssr, volume 269, pages 543-547.

Razzaque, M. A., Bleakley, C., and Dobson, S. (2013). Compression in wireless sensor networks: A survey and comparative evaluation. ACM Transactions on Sensor Networks (TOSN), 10(1):1-44.

Schoellhammer, T., Greenstein, B., Osterweil, E., Wimbrow, M., and Estrin, D. (2004). Lightweight temporal compression of microclimate datasets [wireless sensor networks]. In 29th Annual IEEE International Conference on Local Computer Networks, pages 516-524.

Smith, L. N. (2017). Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 464-472. IEEE.

Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., and Cottrell, G. (2018). Understanding convolution for semantic segmentation. In 2018 IEEE winter conference on applications of computer vision (WACV), pages 1451-1460. Ieee.

Yu, F. and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122.

Yu, T., Wang, X., and Shami, A. (2018). Uav-enabled spatial data sampling in large-scale iot systems using denoising autoencoder neural network. IEEE Internet of Things Journal, 6(2):1856-1865.