LLM optimization in embedded hardware: a systematic review

• Lúcio Vagner Carvalho UFPel
• Carlos Henrique Leite Bianchin UFPel
• Marcos Lima Alves UFPel
• João Carrett UFPel
• Lucas Machado Dias UFPel
• Fabricio Barbosa Viegas UFPel
• Anderson Priebe Ferrugem UFPel

DOI: https://doi.org/10.5753/eniac.2026.29974

Resumo

A implantação de Large Language Models (LLMs) na borda esbarra no gargalo de Von Neumann e em severas restrições energéticas e de memória. Para mapear as soluções a este desafio, esta Revisão Sistemática da Literatura analisou 106 estudos primários extraídos do IEEE Xplore e Scopus. Os resultados revelam a dominância da quantização (55,7%), impulsionada pelo co-design com FPGAs e NPUs, como a estratégia mais eficaz para contornar o tráfego massivo de dados. O estudo evidencia lacunas críticas na otimização da fase de prefill e a sub-representação do ecossistema RISC-V, delineando um roteiro arquitetural para futuras pesquisas em Edge AI.

Referências

Backus, J. (1978). Can programming be liberated from the von neumann style?: a functional style and its algebra of programs. Communications of the ACM, 21(8):613–641.

Booth, A., Sutton, A., and Papaioannou, D. (2016). Systematic Approaches to a Successful Literature Review. Sage Publications, London, UK, 2nd edition.

Cao, J., Lin, X., Zhang, M., Shi, K., Yu, J., and Wang, K. (2023). PP-Transformer: Enable efficient deployment of transformers through pattern pruning. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE/ACM.

Kim, J., Lee, J., Lin, J., Kanani, A., Miao, S., Ogras, U., and Park, J. (2025). eMamba: Efficient acceleration framework for mamba models in edge computing. ACM Transactions on Embedded Computing Systems.

Liu, F., Yang, N., Wang, Z., Zhu, X., Yao, H., Xiong, X., Sun, Q., and Jiang, L. (2025). OPS: Outlier-aware precision-slice framework for LLM acceleration. In Proceedings of Design, Automation and Test in Europe (DATE). IEEE.

Ma, W., Yang, X., Zeng, S., Liu, T., Shen, L., Wang, H., Li, S., Wang, J., Zhang, Y., Guo, H., Li, J., Zhang, Z., Zhu, Z., Ning, X., Ho, T.-Y., Dai, G., and Wang, Y. (2025). FMC-LLM: Enabling FPGAs for efficient batched decoding of 70b+ LLMs with a memory-centric streaming architecture. In Proceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA). ACM.

Qiao, Y., Chen, Z., Zhang, Y., Wang, Y., and Huang, S. (2026). TeLLMe: An efficient end-to-end ternary LLM prefill and decode accelerator with table-lookup matmul on edge FPGAs. In Proceedings of the 2026 ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA). ACM.

Raha, A., Kundu, S., Sridhar, S. N., Kundu, S., Ghosh, S. K., Palla, A., Das, A., Crews, D., and Mathaikutty, D. A. (2025). LLM-NPU: Towards efficient foundation model inference on low-power neural processing units. In 2025 IEEE International Conference on Omni-Layer Intelligent Systems (COINS). IEEE.

Sadheerthan, S., Hsu, M.-J., Huang, C.-H., and Wang, Y.-T. (2026). Attention distribution-aware softmax for NPU-accelerated on-device inference of LLMs: An edge-oriented approximation design. Electronics, 15(6):1312.

Tang, J., Guo, C., Leng, J., Liu, Y., Zhang, C., Guo, M., Hu, W., Yang, F., and Zhu, Y. (2023). OliVe: Accelerating large language models via hardware-friendly outlier-victim pair quantization. In Proceedings of the 50th Annual International Symposium on Computer Architecture (ISCA). ACM.

Wei, C.-C., Wang, C.-H., Pong, E.-Y., and Chen, C.-H. (2025). Low-cost non-linear function approximation for transformer deployment on edge NPU. In 2025 1st International Conference on Consumer Technology (ICCT-Pacific). IEEE.

Wu, J., Zheng, Z., Gong, L., Wang, C., and Zhou, X. (2026). UDP: A universal DSP packing framework for low-bitwidth MAC acceleration on FPGAs. In Proceedings of the 2026 ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA). ACM.

Xu, D., Zhang, H., Yang, L., Liu, R., Xu, M., and Liu, X. (2024a). WiP: Efficient LLM prefilling with mobile NPU. In Proceedings of the 2024 Workshop on Edge and Mobile Foundation Models (EdgeFM). ACM.

Xu, H., Li, Y., and Ji, S. (2024b). LlamaF: An efficient Llama2 architecture accelerator on embedded FPGAs. In 2024 IEEE 10th World Forum on Internet of Things (WF-IoT). IEEE.

Yu, Z., Ji, H., Cao, Y., Wu, J., Yan, X., Zheng, L., and Zou, Z. (2025). DuoQ: A DSP utilization-aware and outlier-free quantization for FPGA-based LLMs acceleration. In Proceedings of the 62nd Design Automation Conference (DAC). IEEE.

Zeng, S., Liu, J., Dai, G., Yang, X., Fu, T., Wang, H., Ma, W., Sun, H., Li, S., Huang, Z., Dai, Y., Li, J., Wang, Z., Zhang, R., Wen, K., Ning, X., and Wang, Y. (2024). FlightLLM: Efficient large language model inference with a complete mapping flow on FPGAs. In Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA). ACM.

Zhang, Y., Wang, H., Zhang, G., Chen, J., Yu, J., and Wang, K. (2025). SiST: Token similarity and sparsity aware optimization for transformers on FPGA. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE/ACM.