Toward Linear Representations of Foundation Models for Histopathology Image Retrieval

• Matheus Santos UFBA
• Igor Borja UFBA
• David Lima UFBA
• Grinaldo Oliveira UFBA
• Ângelo Duarte UEFS
• Washington Santos FIOCRUZ
• Luciano Oliveira UFBA / FIOCRUZ

Resumo

While domain-specific foundation models have accelerated advancements by leveraging large-scale histopathological datasets, they often struggle to generalize across diverse clinical scenarios and datasets. In this paper, we address this limitation with a linear-prototype framework: A 128-D projection head trained by a modified prototypical loss on a few labelled slides. Evaluated on proprietary glomerular biopsies, dermoscopic skin cancer, and ovarian-cancer WSIs, our proposal approach consistently boosts image retrieval. Across nine backbones (UNI/UNI2h, Phikon/Phikon-v2, Virchow2, DINO/DINOv2, ViT, and a ResNet-50 baseline), few-shot tuning raises mAP by up to 15 percentage points. These gains show that our lightweight, prototype-based layer can reconcile the breadth of foundation pre-training with the depth of task-specific discrimination, enabling improvement on medical-image retrieval across diverse pathologies.

Palavras-chave: Graphics, Head, Foundation models, Histopathology, Image retrieval, Biopsy, Proposals, Tuning, Skin cancer, Biomedical imaging