Comparing the Performance of Intel AI Boost NPU with a General-Purpose CPU

• Kenichi Brumati UFRGS
• Igor Freitas Intel
• Arthur F. Lorenzon UFRGS

DOI: https://doi.org/10.5753/eradrs.2025.6810

Abstract

The growing demand for Artificial Intelligence applications has driven the development of specialized hardware, such as Neural Processing Units (NPUs), designed to optimize training and inference tasks. With their increasing adoption in modern processors and embedded devices, it becomes essential to evaluate their impact on application performance. Thus, in this work, we analyze the performance of the Intel AI Boost NPU compared to a general-purpose processor, considering metrics such as latency, inference time, and throughput.

Keywords: Evaluation, Measurement, and Performance Prediction, Heterogeneous Computing

References

Ishida, K., Byun, I., Nagaoka, I., Fukumitsu, K., Tanaka, M., Kawakami, S., Tanimoto, T., Ono, T., Kim, J., and Inoue, K. (2020). Supernpu: An extremely fast neural processing unit using superconducting logic devices. In Annual IEEE/ACM MICRO, pages 58–72.

Kim, S., Kim, S., Hong, S., Kim, S., Han, D., Choi, J., and Yoo, H.-J. (2023). Cool-npu: Complementary online learning neural processing unit with cnn-snn heterogeneous core and event-driven backpropagation. In IEEE COOL CHIPS, pages 1–3.

Wu, B., Furtner, W., Waschneck, B., and Mayr, C. (2022). Industry-track: Towards agile design of neural processing unit. In CODES+ISSS, pages 17–20.

Xue, Y., Liu, Y., and Huang, J. (2023). System virtualization for neural processing units. In HOTOS, page 80–86.

Zhang, W., Peng, X., Wu, H., Gao, B., He, H., Zhang, Y., Yu, S., and Qian, H. (2019). Design guidelines of rram based neural-processing-unit: A joint device-circuit-algorithm analysis. In ACM/IEEE DAC, pages 1–6.