Evaluating Lightweight Transformers for Network Intrusion Detection under Temporal Drift: A Comparative Study with MAWIFlow - Jardim Neto, João; Marcondes, César Augusto Cavalheiro (Anais do Simpósio Brasileiro de Cibersegurança (SBSeg); 2026: Anais do XXVI Simpósio Brasileiro de CibersegurançaDO - 10.5753/sbseg.2026.27067, 2026)
Static train/test splits remain the dominant evaluation protocol for network intrusion detection systems (NIDS), yet real-world traffic evolves continuously and deployed models degrade under temporal drift. We present a reproducible comparative evaluation of four model families—XGBoost, MLP, CNN-BiLSTM, and a parameter-efficient causal Transformer (LightTransformer, a compact FlowTransformer-derived configuration)—on two complementary protocols: a static split on CICIoT2023 (sanity check) and a temporal forward-chaining evaluation on MAWIFlow (2007–2024). On the static benchmark all models achieve F1 + >0.98 on the attack class, so near-ceiling static scores are insufficient evidence of temporal robustness. Under temporal drift (cumulative training window, R=5 seeds, all models) the LightTransformer obtains nAUT 1 =0.57 (95% CI [0.52, 0.62]) versus 0.44 (CNN-BiLSTM), 0.38 (XGBoost), and 0.28 (MLP). Bootstrap confidence intervals are consistent with this ordering, which holds across all five seeds; a Friedman test (χ 2 =15.0, p=0.002) rejects equal rankings globally; one-sided pairwise Wilcoxon signed-rank tests provide uncorrected directional evidence (p=0.031 for all six pairs, n=5); and no pairwise comparison survives Bonferroni correction (α c =0.0083). Our results indicate that forward-chaining temporal evaluation should be a standard component of deployment-oriented NIDS benchmarking.