Toward Linear Representations of Foundation Models for Histopathology Image Retrieval - Santos, Matheus; Borja, Igor; Lima, David; Oliveira, Grinaldo; Duarte, Ângelo; Santos, Washington; Oliveira, Luciano (Anais da Conference on Graphics, Patterns and Images (SIBGRAPI); 2025: Anais da XXXVIII Conference on Graphics, Patterns and Images, 2025)
While domain-specific foundation models have accelerated advancements by leveraging large-scale histopathological datasets, they often struggle to generalize across diverse clinical scenarios and datasets. In this paper, we address this limitation with a linear-prototype framework: A 128-D projection head trained by a modified prototypical loss on a few labelled slides. Evaluated on proprietary glomerular biopsies, dermoscopic skin cancer, and ovarian-cancer WSIs, our proposal approach consistently boosts image retrieval. Across nine backbones (UNI/UNI2h, Phikon/Phikon-v2, Virchow2, DINO/DINOv2, ViT, and a ResNet-50 baseline), few-shot tuning raises mAP by up to 15 percentage points. These gains show that our lightweight, prototype-based layer can reconcile the breadth of foundation pre-training with the depth of task-specific discrimination, enabling improvement on medical-image retrieval across diverse pathologies.