DiagHW: A Compact LLM for Hardware Failure Diagnosis via a Novel Knowledge Distillation Pipeline

• Carlos Caminha UFC
• Maria de Lourdes M. Silva UFC
• Iago C. Chaves UFC
• Felipe T. Brito UFC
• Victor A. E. Farias UFC
• Javam C. Machado UFC

Resumo

Hardware failure diagnosis from textual user-reported issues presents significant challenges due to the ambiguous and non-technical nature of these reports. While Large Language Models show promise in this domain, state-of-the-art models with billions of parameters pose practical limitations for deployment on low-power consumer devices. This work introduces a novel knowledge distillation pipeline for hardware failure detection, leveraging a large teacher model to generate synthetic training data for a smaller student model. Then, we present the DiagHW model, a compact 1.2B parameter fine-tuned LLaMA-3.2-1b-instruct model, which achieves diagnostic accuracy comparable to much larger models (up to 72B parameters). Our extensive experimental validation involved 32,414 inferences across 28 baseline models and 10 fine-tuned variants, processing a total of 19,692,210 tokens for these evaluations, complemented by 105,925,876 tokens processed during the fine-tuning stages.