Uma Arquitetura de Rede Neural com Auxílio da Nuvem para Dispositivos Computacionalmente Limitados
Resumo
A execução de redes neurais profundas (Deep Neural Networks – DNNs) pode ser inviável em dispositivos de baixo poder computacional. Este trabalho propõe um modelo de rede neural otimizado para esses dispositivos, ao custo de uma menor acurácia, mas auxiliado por uma DNN na nuvem caso a inferência não atinja um determinado valor de confiança. Experimentos com dígitos escritos à mão mostram que o modelo ocupa baixa quantidade de memória, além de possuir tempo de treinamento e de inferência menores quando comparado a uma DNN conhecida. Adicionalmente, a estratégia de realizar a inferência local antes de consultar a nuvem reduziu o tempo médio de inferência em pelo menos quatro vezes, com uma acurácia de 96%.
Referências
Aleksander, I., De Gregorio, M., França, F. M. G., Lima, P. M. V. e Morton, H. (2009). A brief introduction to weightless neural systems. Em European Symposium on Artificial Neural Networks Advances in Computational Intelligence and Learning (ESANN), p. 299–305. Citeseer.
Aleksander, I., Thomas, W. e Bowden, P. (1984). Wisard· a radical step forward in image recognition. Sensor review, 4(3):120–124.
Bloom, B. H. (1970). Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13:422–426.
Bochie, K., Gilbert, M. S., Gantert, L., Barbosa, M. S. M., Medeiros, D. S. V. e Campista, M. E. M. (2021). A survey on deep learning for challenged networks: Applications and trends. Journal of Network and Computer Applications, 194:103213.
Chen, X., Zhang, J., Lin, B., Chen, Z., Wolter, K. e Min, G. (2021). Energy-efficient offloading for dnn-based smart iot systems in cloud-edge environments. IEEE Transactions on Parallel and Distributed Systems, 33(3):683–697.
Grieco, B. P., Lima, P. M., De Gregorio, M. e França, F. M. (2010). Producing pattern examples from “mental” images. Neurocomputing, 73(7):1057–1064.
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R. e Bengio, Y. (2016). Binarized neural networks. Advances in Neural Information Processing Systems, 29.
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R. e Bengio, Y. (2017). Quantized neural networks: Training neural networks with low precision weights and activations. The Journal of Machine Learning Research, 18(1):6869–6898.
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H. e Kalenichenko, D. (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Em Proceedings of the IEEE Conference on Computer Vision and Patter Recognition, p. 2704–2713.
LeCun, Y., Jackel, L. D., Bottou, L., Cortes, C., Denker, J. S., Drucker, H., Guyon, I., Muller, U. A., Sackinger, E., Simard, P. et al. (1995). Learning algorithms for classification: A comparison on handwritten digit recognition. Neural Networks: The Statistical Mechanics Perspective, 261(276):2.
Matsubara, Y., Levorato, M. e Restuccia, F. (2022). Split computing and early exiting for deep learning applications: Survey and research challenges. ACM Computing Surveys, 55(5):1–30.
McDanel, B., Teerapittayanon, S. e Kung, H. T. (2017). Embedded binarized neural networks. arXiv preprint arXiv:1709.02260.
Pacheco, R., Couto, R. e Simeone, O. (2021a). Calibration-aided edge inference offloading via adaptive model partitioning of deep neural networks. Em IEEE International Conference on Communications (ICC), p. 1–6.
Pacheco, R., Oliveira, F. R. e Couto, R. (2021b). Early-exit deep neural networks for distorted images: providing an efficient edge offloading. Em IEEE Global Communications Conference (GLOBECOM), p. 1–6.
Rajapakse, V., Karunanayake, I. e Ahmed, N. (2022). Intelligence at the extreme edge: a survey on reformable tinyml. ACM Computing Surveys.
Rastegari, M., Ordonez, V., Redmon, J. e Farhadi, A. (2016). Xnor-net: Imagenet classification using binary convolutional neural networks. CoRR, abs/1603.05279.
Santiago, L., Verona, L., Rangel, F., Firmino, F., Menasché, D. S., Caarls, W., Breternitz Jr, M., Kundu, S., Lima, P. M. e França, F. M. (2020). Weightless neural networks as memory segmented bloom filters. Neurocomputing, 416:292–304.
Susskind, Z., Arora, A., Miranda, I. D. D. S., Villon, L. A. Q., Katopodis, R. F., de Araújo, L. S., Dutra, D. L. C., Lima, P. M. V., Franca, F. M. G., Breternitz Jr, M. et al. (2022). Weightless neural networks for efficient edge inference. arXiv preprint arXiv:2203.01479.
Teerapittayanon, S., McDanel, B. e Kung, H. (2017). Distributed deep neural networks over the cloud, the edge and end devices. Em 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), p. 328–339.
Teerapittayanon, S., McDanel, B. e Kung, H.-T. (2016). Branchynet: Fast inference via early exiting from deep neural networks. Em IEEE International Conference on Pattern Recognition (ICPR), p. 2464–2469.
Torres, V. A., Jaimes, B. R., Ribeiro, E. S., Braga, M. T., Shiguemori, E. H., Velho, H. F., Torres, L. C. e Braga, A. P. (2020). Combined weightless neural network FPGA architecture for deforestation surveillance and visual navigation of uavs. Engineering Applications of Artificial Intelligence, 87:103227.
Xue, M., Wu, H., Peng, G. e Wolter, K. (2021). DDPQN: An efficient dnn offloading strategy in local-edge-cloud collaborative environments. IEEE Transactions on Services Computing, 15(2):640–655.
Aleksander, I., Thomas, W. e Bowden, P. (1984). Wisard· a radical step forward in image recognition. Sensor review, 4(3):120–124.
Bloom, B. H. (1970). Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13:422–426.
Bochie, K., Gilbert, M. S., Gantert, L., Barbosa, M. S. M., Medeiros, D. S. V. e Campista, M. E. M. (2021). A survey on deep learning for challenged networks: Applications and trends. Journal of Network and Computer Applications, 194:103213.
Chen, X., Zhang, J., Lin, B., Chen, Z., Wolter, K. e Min, G. (2021). Energy-efficient offloading for dnn-based smart iot systems in cloud-edge environments. IEEE Transactions on Parallel and Distributed Systems, 33(3):683–697.
Grieco, B. P., Lima, P. M., De Gregorio, M. e França, F. M. (2010). Producing pattern examples from “mental” images. Neurocomputing, 73(7):1057–1064.
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R. e Bengio, Y. (2016). Binarized neural networks. Advances in Neural Information Processing Systems, 29.
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R. e Bengio, Y. (2017). Quantized neural networks: Training neural networks with low precision weights and activations. The Journal of Machine Learning Research, 18(1):6869–6898.
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H. e Kalenichenko, D. (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Em Proceedings of the IEEE Conference on Computer Vision and Patter Recognition, p. 2704–2713.
LeCun, Y., Jackel, L. D., Bottou, L., Cortes, C., Denker, J. S., Drucker, H., Guyon, I., Muller, U. A., Sackinger, E., Simard, P. et al. (1995). Learning algorithms for classification: A comparison on handwritten digit recognition. Neural Networks: The Statistical Mechanics Perspective, 261(276):2.
Matsubara, Y., Levorato, M. e Restuccia, F. (2022). Split computing and early exiting for deep learning applications: Survey and research challenges. ACM Computing Surveys, 55(5):1–30.
McDanel, B., Teerapittayanon, S. e Kung, H. T. (2017). Embedded binarized neural networks. arXiv preprint arXiv:1709.02260.
Pacheco, R., Couto, R. e Simeone, O. (2021a). Calibration-aided edge inference offloading via adaptive model partitioning of deep neural networks. Em IEEE International Conference on Communications (ICC), p. 1–6.
Pacheco, R., Oliveira, F. R. e Couto, R. (2021b). Early-exit deep neural networks for distorted images: providing an efficient edge offloading. Em IEEE Global Communications Conference (GLOBECOM), p. 1–6.
Rajapakse, V., Karunanayake, I. e Ahmed, N. (2022). Intelligence at the extreme edge: a survey on reformable tinyml. ACM Computing Surveys.
Rastegari, M., Ordonez, V., Redmon, J. e Farhadi, A. (2016). Xnor-net: Imagenet classification using binary convolutional neural networks. CoRR, abs/1603.05279.
Santiago, L., Verona, L., Rangel, F., Firmino, F., Menasché, D. S., Caarls, W., Breternitz Jr, M., Kundu, S., Lima, P. M. e França, F. M. (2020). Weightless neural networks as memory segmented bloom filters. Neurocomputing, 416:292–304.
Susskind, Z., Arora, A., Miranda, I. D. D. S., Villon, L. A. Q., Katopodis, R. F., de Araújo, L. S., Dutra, D. L. C., Lima, P. M. V., Franca, F. M. G., Breternitz Jr, M. et al. (2022). Weightless neural networks for efficient edge inference. arXiv preprint arXiv:2203.01479.
Teerapittayanon, S., McDanel, B. e Kung, H. (2017). Distributed deep neural networks over the cloud, the edge and end devices. Em 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), p. 328–339.
Teerapittayanon, S., McDanel, B. e Kung, H.-T. (2016). Branchynet: Fast inference via early exiting from deep neural networks. Em IEEE International Conference on Pattern Recognition (ICPR), p. 2464–2469.
Torres, V. A., Jaimes, B. R., Ribeiro, E. S., Braga, M. T., Shiguemori, E. H., Velho, H. F., Torres, L. C. e Braga, A. P. (2020). Combined weightless neural network FPGA architecture for deforestation surveillance and visual navigation of uavs. Engineering Applications of Artificial Intelligence, 87:103227.
Xue, M., Wu, H., Peng, G. e Wolter, K. (2021). DDPQN: An efficient dnn offloading strategy in local-edge-cloud collaborative environments. IEEE Transactions on Services Computing, 15(2):640–655.
Publicado
26/05/2023
Como Citar
MEDEIROS, Caio Gevegir Miguel; CRUZ, Pedro; COUTO, Rodrigo de Souza.
Uma Arquitetura de Rede Neural com Auxílio da Nuvem para Dispositivos Computacionalmente Limitados. In: WORKSHOP DE GERÊNCIA E OPERAÇÃO DE REDES E SERVIÇOS (WGRS), 28. , 2023, Brasília/DF.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 1-14.
ISSN 2595-2722.
DOI: https://doi.org/10.5753/wgrs.2023.741.
