On the benefits of automated tuning of hyper-parameters: an experiment related to temperature prediction on UAV computers

• Renato de Sousa Maximiano INPE
• Valdivino Alexandre de Santiago Júnior INPE
• Elcio Hideiti Shiguemori IEAV

DOI: https://doi.org/10.5753/eniac.2022.227276

Resumo

Finding the best configuration of a neural network to solve a problem has been challenging given the numerous possibilities of values of the hyper-parameters. Thus, tuning of hyper-parameters is one important approach and researchers suggest doing this automatically. However, it is important to verify when it is suitable to perform automated tuning which is usually very costly financially and also in terms of hardware infrastructure. In this study, we analyze the advantages of using a hyper-parameter optimization framework as a way of optimizing the automated search for hyper-parameters of a neural network. To achieve this goal, we used data from an experiment related to temperature prediction of computers embedded in unmanned aerial vehicles (UAVs), and the models Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) to perform these predictions. In addition, we compare the hyper-parameter optimization framework to the hyper-parameter exhaustive search technique varying the size of the training dataset. Results of our experiment shows that designing a model using a hyper-parameter optimizer can be up to 36.02% better than using exhaustive search, in addition to achieving satisfactory results with a reduced dataset.

Referências

Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019). Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623-2631.

AlexeyAB (2022). https://github.com/AlexeyAB/darknet. [Last accessed: 4 JUN 2022].

Amato, G., Falchi, F., Gennaro, C., Massoli, F. V., and Vairo, C. (2020). Multi-resolution face recognition with drones. In 2020 3rd International Conference on Sensors, Signal and Image Processing, pages 13-18.

Bandopadhyay, D., Jha, V., Bandyopadhyay, A., Roy, P., Halder, R., and Majhi, S. (2022). Automated people monitoring system using opencv and raspberry pi. In ICT Analysis and Applications, pages 905-913. Springer.

Benoit-Cattin, T., Velasco-Montero, D., and Fernández-Berni, J. (2020). Impact of thermal throttling on long-term visual inference in a cpu-based edge device. Electronics, 9(12):2106.

Cañar, R. L., Fontaine, A., Morillo, P. L., and El Yacoubi, S. (2020). Deep learning to implement a statistical weather forecast for the andean city of quito. In 2020 IEEE ANDESCON, pages 1-6. IEEE.

Castellano, G., Castiello, C., Mencar, C., and Vessio, G. (2020). Preliminary evaluation of tinyyolo on a new dataset for search-and-rescue with drones. In 2020 7th International Conference on Soft Computing&Machine Intelligence (ISCMI), pages 163-166. IEEE.

Chui, K. T., Gupta, B. B., and Vasant, P. (2021). A genetic algorithm optimized rnn-lstm model for remaining useful life prediction of turbofan engine. Electronics, 10(3):285.

Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. cocodataset (2022). https://cocodataset.org/#home. [Last accessed: 08 FEB 2022].

de Caux, M., Bernardini, F., and Viterbo, J. (2020). Short-term forecasting in bitcoin time series using lstm and gru rnns. In Anais do VIII Symposium on Knowledge Discovery, Mining and Learning, pages 97-104. SBC.

Dji (2022). https://www.dji.com/br. [Last accessed: 25 FEB 2022].

dos Santos Antoniassi, R. A. (2022). Predição de nível de rios da região hidrográfica do rio paraguai utilizando algoritmos de aprendizado de máquina.

Ekundayo, I. (2020a). Optuna optimization based cnn-lstm model for predicting electric power consumption. PhD thesis, Dublin, National College of Ireland.

Ekundayo, I. (2020b). Optuna optimization based cnn-lstm model for predicting electric power consumption. PhD thesis, Dublin, National College of Ireland.

Elsworth, S. and Güttel, S. (2020). Time series forecasting using lstm networks: A symbolic approach. arXiv preprint arXiv:2003.05672.

escoladeestudantes (2022). [link]. [Last accessed: 4 JUN 2022].

Hamida, S., El Gannour, O., Cherradi, B., Ouajji, H., and Raihani, A. (2020). Optimization of machine learning algorithms hyper-parameters for improving the prediction of patients infected with covid-19. In 2020 ieee 2nd international conference on electronics, control, optimization and computer science (icecocs), pages 1-6. IEEE.

Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8):1735-1780.

Joseph, F. J. J. (2019). Iot based weather monitoring system for effective analytics. International Journal of Engineering and Advanced Technology, 8(4):311-315.

Kinaneva, D., Hristov, G., Raychev, J., and Zahariev, P. (2019). Early forest fire detection using drones and artificial intelligence. In 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 1060-1065. IEEE.

Kong, W., Dong, Z. Y., Luo, F., Meng, K., Zhang, W., Wang, F., and Zhao, X. (2017). Effect of automatic hyperparameter tuning for residential load forecasting via deep learning. In 2017 australasian universities power engineering conference (aupec), pages 1-6. IEEE.

lncc (2022). https://sdumont.lncc.br/. [Last accessed: 4 JUN 2022].

Machowski, J. and Dziénkowski, M. (2021). Selection of the type of cooling for an overclocked raspberry pi 4b minicomputer processor operating at maximum load conditions. Journal of Computer Sciences Institute, 18:55-60.

Manganiello, F. (2021). Computer vision on raspberry pi. In Computer Vision with Maker Tech, pages 159-225. Springer.

Moghar, A. and Hamiche, M. (2020). Stock market prediction using lstm recurrent neural network. Procedia Computer Science, 170:1168-1173.

Munawar, H. S., Ullah, F., Heravi, A., Thaheem, M. J., and Maqsoom, A. (2021). Inspecting buildings using drones and computer vision: A machine learning approach to detect cracks and damages. Drones, 6(1):5.

Nishitsuji, Y. and Nasseri, J. (2022). Lstm with forget gates optimized by optuna for lithofacies prediction.

Parsa, M., Mitchell, J. P., Schuman, C. D., Patton, R. M., Potok, T. E., and Roy, K. (2019). Bayesian-based hyperparameter optimization for spiking neuromorphic systems. In 2019 IEEE International Conference on Big Data (Big Data), pages 4472-4478. IEEE.

Patidar, S., Jindal, R., and Kumar, N. (2021). Streamed covid-19 data analysis using lstm-a deep learning technique. In Soft Computing for Problem Solving, pages 493-504. Springer.

Prathaban, T., Thean, W., and Sazali, M. I. S. M. (2019). A vision-based home security system using opencv on raspberry pi 3. In AIP Conference Proceedings, volume 2173, page 020013. AIP Publishing LLC.

Raspberry-Pi (2022). https://www.raspberrypi.com/products/raspberry-pi-4-model-b/. [Last accessed: 25 FEB 2022].

Tukymbekov, D., Saymbetov, A., Nurgaliyev, M., Kuttybay, N., Dosymbetova, G., and Svanbayev, Y. (2021). Intelligent autonomous street lighting system based on weather forecast using lstm. Energy, 231:120902.

Turner, R., Eriksson, D., McCourt, M., Kiili, J., Laaksonen, E., Xu, Z., and Guyon, I. (2021). Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. In NeurIPS 2020 Competition and Demonstration Track, pages 3-26. PMLR.

Zegarra, F. C., Vargas-Machuca, J., and Coronado, A. M. (2021). Comparison of cnn and cnn-lstm architectures for tool wear estimation. In 2021 IEEE Engineering International Research Conference (EIRCON), pages 1-4. IEEE.