Dataset Inference in Fine-Tuned Large Language Models: A Comparative Study

• Gabriel Vaz de Oliveira UFC
• Arthur Santos Viana de Oliveira UFC
• Victor Aguiar Evangelista de Farias UFC
• Javam de Castro Machado UFC
• Carlos Caminha UFC

DOI: https://doi.org/10.5753/semish.2026.23761

Resumo

Dataset Inference provides a robust framework for auditing data ownership by aggregating statistical signals that traditional Membership Inference Attacks (MIAs) fail to capture. While proven for LLM pre-training, its efficacy during fine-tuning is largely unexplored. We evaluate Dataset Inference on Gemma, Llama, and Qwen models using Full Fine-Tuning (FFT), LoRA, and QLoRA. Our findings reveal a stark architectural divergence: ParameterEfficient Fine-Tuning (PEFT) mitigates data leakage in Llama (AUC ≈ 0.50, p > 0.05), while Gemma and Qwen remain highly vulnerable across all adaptation methods (p < 0.05). Additionally, higher learning rates accelerate data absorption, and QLoRA provides only marginal regularization compared to standard LoRA.

Referências

Das, B. C., Amini, M. H., and Wu, Y. (2025). Security and privacy challenges of large language models: A survey. ACM Computing Surveys, 57(6):1–39.

Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023). Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088–10115.

Duan, M., Suri, A., Mireshghallah, N., Min, S., Shi, W., Zettlemoyer, L., Tsvetkov, Y., Choi, Y., Evans, D., and Hajishirzi, H. (2024). Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841.

Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. (2022). Lora: Low-rank adaptation of large language models. Iclr, 1(2):3.

Jagannatha, A., Rawat, B. P. S., and Yu, H. (2021). Membership inference attack susceptibility of clinical language models. arXiv preprint arXiv:2104.08305.

Maini, P., Jia, H., Papernot, N., and Dziedzic, A. (2024). Llm dataset inference: Did you train on my dataset? Advances in Neural Information Processing Systems, 37:124069–124092.

R3troR0b (2024). News-dataset. [link]. [Dataset]. Accessed: 2026-03-02.

Raschka, S. (2024). Build a large language model (from scratch). Simon and Schuster.