MulitaMiner: A Multi-Version Evaluation of LLM-Based Vulnerability Report Extraction - Machado, Beatriz; Lautert, Douglas; Kapelinski, Cristhian; Kreutz, Diego; Ferrão, Isadora Garcia; Bof, Alessandro (Anais do Simpósio Brasileiro de Cibersegurança (SBSeg); 2026: Anais do XXVI Simpósio Brasileiro de Cibersegurança, 2026)
Security teams face 131 new CVEs per day, yet the National Vulnerability Database fails to enrich 44% of new submissions, leaving practitioners without the structured metadata required to prioritize remediation. MulitaMiner closes this gap: an LLM-based pipeline for structured vulnerability extraction from heterogeneous scanner PDFs, hardened across three successive versions (V1–V3) that evolve from a functional baseline, to scanner-aware segmentation, and finally to per-LLM tuning with few-shot prompting. We evaluate five LLMs (DeepSeek, GPT-4, GPT-5, LLaMA 3, LLaMA 4) on three manually curated OpenVAS baselines across 450 independent runs. Without changing the underlying models, Exact Record Match rises from 37.5% to 90.4%, vulnerability-level omission falls by an order of magnitude (20.5% → 1.7%), and cross-model variance in field-level omission collapses from 41.8 to 2.9 percentage points, all with non-overlapping 95% bootstrap confidence intervals. These results establish that pipeline engineering, not model substitution, is the primary lever for extraction quality: within the scope evaluated here, a well-engineered pipeline turns LLM choice into a question of cost and latency rather than accuracy.