Comparison of YOLO instance segmentation models in identifying forest species using leaf images - Osowski, Isabela Yasmim; Fadel, Eduardo Marcon Gonçalves; Cielo Filho, Roque; Sabbi, Larissa De Bortolli Chiamolera; Paula Filho, Pedro Luiz de (Anais do Congresso Latino-Americano de Software Livre e Tecnologias Abertas (Latinoware); 2025: Anais do XXII Congresso Latino-Americano de Software Livre e Tecnologias AbertasDO - 10.5753/latinoware.2025.16492, 2025)
The vast area of forests in Brazil, along with the biodiversity of the flora, poses a challenge to the identification of tree species. This process can be time-sensitive due to the nature of certain projects; therefore, the use of leaves for the identification is ideal in such cases. In order to assist this process, automated tools that use deep learning can be an aid in the conservation efforts. This study evaluates and compares two YOLO (You Only Look Once) models, the nano versions of YOLOv11 and YOLOv12, for the task of identifying species using their leaves. For this, a custom dataset was created with 26 species under varied conditions; the images also have a complex background to guarantee the model’s efficiency in fieldwork. Overall, YOLOv11 achieved higher recall (56.746%) and mAP@50 (72.634%), YOLOv12 demonstrated superior precision (86.054%) and better per-class performance. The results show that both models’ performance varies due to the dataset annotation and the leaf morphology, as both models struggled with species exhibiting compound leaves. This work contributes to the development of practical tools for biodiversity monitoring and supports future advancements in automated species identification, highlighting the potential of lightweight models for tree species identification.