Immersive 3D Manifolds from Deep Embeddings: Evaluation of Image Classifications in VR

• Meaghan Boykin Sonoma State University
• Guilherme Melo dos Santos UNIFESP
• Carlos Gurjão de Godoy UNIFESP
• Regina C. Coelho UNIFESP
• Daniela Ushizima Lawrence Berkeley National Laboratory / University of California San Francisco

DOI: https://doi.org/10.5753/sbcas.2026.21302

Resumo

We show that our Meta Quest 3/3S application provides an immersive environment for the exploration of 3D manifolds generated from deep learning embeddings. Built with Unreal Engine 5.5.4, our tool enables the evaluation and sorting of biomedical specimens classifications by projecting high-dimensional semantic features into a navigable 3D space. Within the virtual environment, users can validate automated results by interactively categorizing images into defined spatial clusters, with the ability to export refined datasets via CSV. This systematic workflow improves quality control for large-scale imaging, offering researchers an intuitive interface to rapidly sort complex biological datasets.

Referências

Alegro, M., Theofilas, P., Nguy, A., Castruita, P. A., Seeley, W., Heinsen, H., Ushizima, D. M., and Grinberg, L. T. (2017). Automating cell detection and classification in human brain fluorescent microscopy images using dictionary learning and sparse coding. Journal of neuroscience methods, 282:20–33.

Barboza, L., Silva, F., Oliveira, H., Oliveira, J., and Brasil, A. (2025). Supporting stroke rehabilitation through gamified daily activity simulations. In XXV SBCAS. SBC.

Ester, M., Kriegel, H.-P., Sander, J., and Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD, pages 226–231.

He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 770–778.

McInnes, L., Healy, J., and Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. Journal of Open Source Software, 3(29):861.

Pandolfi, R. J., Todd, J., Donatelli, J., and Ushizima, D. (2026). Ascribe-xr: Extended reality for visualization of scientific images. In Proc. 11th Int. Wrkshp on Data Analysis and Reduction for Big Scientific Data (DRBSD-11) at Super Computing (SC26).

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019). Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 32.

Ramalho, G. L., Ferreira, D. S., Bianchi, A. G., Carneiro, C. M., Medeiros, F. N., and Ushizima, D. M. (2015). Cell reconstruction under voronoi and enclosing ellipses from 3d microscopy. In IEEE International Symposium on Biomedical Imaging (ISBI).

Rezende, M. T., Silva, R., Bernardo, F. d. O., Tobias, A. H. G., Oliveira, P. H. C., Machado, T. M., Costa, C. S., Medeiros, F. N. S., Ushizima, D. M., Carneiro, C. M., and Bianchi, A. G. C. (2021). Cric searchable image database as a public platform for conventional pap smear cytology data. Nature Scientific Data, 8(1):151.

Ushizima, D., dos Santos, G. M., Sordo, Z., Pandolfi, R., and Donatelli, J. (2025). Ascribe new dimensions to scientific data visualization and interactive exploration with VR. DOI: 10.48550/arXiv.2504.13448.

Ushizima, D., Xu, K., and Monteiro, P. J. (2020). Materials data science for microstructural characterization of archaeological concrete. MRS Advances, 5(7):305–318.

van der Stappen, A. and Funk, M. (2021). Towards guidelines for designing human-in-the-loop machine training interfaces. In Proceedings of the 26th International Conference on Intelligent User Interfaces, IUI ’21, page 514–519, New York, NY, USA. Association for Computing Machinery.

Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., and He, L. (2022). A survey of human-in-the-loop for machine learning. Future Generation Computer Systems, 135:364–381.