SPINN: a Tool for Distributed Patch Inference on Massive Data Samples

• João Seródio UNICAMP
• Julio C. Faracco UNICAMP
• Fernando Gubitoso UNICAMP
• Otávio O. Napoli UNICAMP
• Alan Souza PETROBRAS
• Daniel Miranda PETROBRAS
• Carlos A. Astudillo UNICAMP
• Edson Borin UNICAMP

Resumo

Patched inference is a widely used technique in machine learning (ML) that enables fixed-shape models to process arbitrarily large or variably sized inputs by dividing them into smaller, compatible patches. This approach is particularly useful in domains such as seismic processing, medical imaging, and electron microscopy, where data samples often exceed the memory capacity of individual computing nodes. While patched inference is effective for leveraging pre-trained models and operating on resource-constrained hardware, there remains a lack of tools supporting its efficient, distributed execution at scale.To address this gap, we introduce SPINN (Scalable Parallel INference Network), a Python library designed to streamline and accelerate patched inference on high-performance computing (HPC) systems. SPINN supports data partitioning, patch-wise processing using user-defined ML models, and result aggregation, all while leveraging distributed computing frameworks such as Dask and Ray.We validate SPINN on two seismic interpretation tasks, fault detection and facies segmentation, using both public and large-scale private data (up to 272 GB). Experiments demonstrate that SPINN enables smoother prediction outputs via overlapping patches and achieves superlinear scalability with Dask in HPC environments, significantly outperforming conventional solutions such as the NVIDIA Triton Inference Server in large-scale scenarios. SPINN thus emerges as a robust and scalable solution for applying deep learning inference to massive data samples in memory-constrained or compute-intensive settings.

Palavras-chave: Deep learning, Computational modeling, High performance computing, Scalability, Fault detection, Distributed databases, Libraries, Hardware, Servers, Electrons, SPINN, Deep Learning, Patched Inference, Distributed Inference, Dask, Ray