VISHOD - Visual Dataset Scraping and Hybrid Outlier Detection
Resumo
Supervised learning depends on data quality, yet manual labeling is costly for resource-limited environments. This paper presents an automated pipeline for building and refining image datasets via web scraping and hybrid outlier detection. The method integrates automated collection, deep learning semantic extraction, consensus-driven anomaly detection, and statistical validation. As a case study, a high-quality dataset without manual annotation was produced for the classification of 2D geometric shapes for educational use in the Amazon. Results show a 25.6% increase in average PCA variance, a 5.6% reduction in centroid distance, and a 21.6% gain in the structural similarity index, evidencing greater diversity, cohesion, and visual homogeneity in the data.Referências
Adnan, M. M., Rahim, M. S. M., Rehman, A., Mehmood, Z., Saba, T., and Naqvi, R. A. (2021). Automatic image annotation based on deep learning models: a systematic review and future challenges. IEEe Access, 9:50253–50264.
Aghabagherloo, A., Abadi, A., Sarkar, S., Dasu, V. A., and Preneel, B. (2025). Impact of data duplication on deep neural network-based image classifiers: Robust vs. standard models. arXiv preprint arXiv:2504.00638.
Al Farizi, W. S., Hidayah, I., and Rizal, M. N. (2021). Isolation forest based anomaly detection: A systematic literature review. In 2021 8th International Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), pages 118–122. IEEE.
Gong, Y., Liu, G., Xue, Y., Li, R., and Meng, L. (2023). A survey on dataset quality in machine learning. Information and Software Technology, 162:107268.
Goyal, M., Lather, Y., and Lather, V. (2015). Analytical relation & comparison of psnr and ssim on babbon image and human eye perception using matlab. International Journal of Advanced Research in Engineering and Applied Sciences, 4(5):108–119.
Guo, L. L., Pfohl, S. R., Fries, J., Posada, J., Fleming, S. L., Aftandilian, C., Shah, N., and Sung, L. (2021). Systematic review of approaches to preserve machine learning performance in the presence of temporal dataset shift in clinical medicine. Applied clinical informatics, 12(04):808–815.
Habuza, T., Navaz, A. N., Hashim, F., Alnajjar, F., Zaki, N., Serhani, M. A., and Statsenko, Y. (2021). Ai applications in robotics, diagnostic image analysis and precision medicine: Current limitations, future trends, guidelines on cad systems for medicine. Informatics in Medicine Unlocked, 24:100596.
Kadhim, Y. A., Khan, M. U., and Mishra, A. (2022). Deep learning-based computer-aided diagnosis (cad): Applications for medical image datasets. Sensors, 22(22):8999.
Kim, T.-G. (2024). Hyperboliclr: Epoch insensitive learning rate scheduler. arXiv preprint arXiv:2407.15200.
Li, X., Li, T., Li, S., Tian, B., Ju, J., Liu, T., and Liu, H. (2023). Learning fusion feature representation for garbage image classification model in human–robot interaction. Infrared Physics & Technology, 128:104457.
Nassif, A. B., Talib, M. A., Nasir, Q., and Dakalbab, F. M. (2021). Machine learning for anomaly detection: A systematic review. Ieee Access, 9:78658–78700.
Rabasovic, M., Pavlovic, D., and Sevic, D. (2023). Analysis of laser ablation spectral data using dimensionality reduction techniques: Pca, t-sne and umap. Contrib. Astron. Obs. Skalnaté Pleso, 53(3):51–57.
Rivas, D. V., Pino, M. M., Pérez, Y. P., Rivas, S. V., Torres, O. G., Fernández, V. C., and Ysa, R. S. (2020). Comparison of distance methods for detection of atypical observations in monthly precipitation series. Revista de Climatología, 20.
Rodríguez-Rodríguez, J. A., López-Rubio, E., Ángel-Ruiz, J. A., and Molina-Cabello, M. A. (2024). The impact of noise and brightness on object detection methods. Sensors, 24(3):821.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022). Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278–25294.
Sharma, P., Ninomiya, T., Omodaka, K., Takahashi, N., Miya, T., Himori, N., Okatani, T., and Nakazawa, T. (2022). A lightweight deep learning model for automatic segmentation and analysis of ophthalmic images. Scientific reports, 12(1):8508.
Stewart, G. and Al-Khassaweneh, M. (2022). An implementation of the hdbscan* clustering algorithm. Applied Sciences, 12(5):2405.
Tan, J., Hou, B., Day, T., Simpson, J., Rueckert, D., and Kainz, B. (2021). Detecting outliers with poisson image interpolation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part V 24, pages 581–591. Springer.
Tan, M. and Le, Q. V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. CoRR, abs/1905.11946.
Thota, P. and Ramez, E. (2021). Web scraping of covid-19 news stories to create datasets for sentiment and emotion analysis. In Proceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference, pages 306–314.
Vieira, N. (2022). Accessibility in the legal amazon: Digital solutions. Climate Policy Initiative.
Wen, L., Gao, L., Li, X., and Zeng, B. (2021). Convolutional neural network with automatic learning rate scheduler for fault classification. IEEE Transactions on Instrumentation and Measurement, 70:1–12.
Xiao, K. and Ni, T. (2024). Computer-aided industrial product design based on image enhancement algorithm and convolutional neural network. Computer-Aided Design and Applications, 21(1).
Xu, Z., Liu, Z., Yan, Y., Liu, Z., Yu, G., and Xiong, C. (2024). Cleaner pretraining corpus curation with neural web scraping. arXiv preprint arXiv:2402.14652.
Zhang, Y., Zhan, Q., and Ma, Z. (2024). Efficientnet-eca: A lightweight network based on efficient channel attention for class-imbalanced welding defects classification. Advanced Engineering Informatics, 62:102737.
Aghabagherloo, A., Abadi, A., Sarkar, S., Dasu, V. A., and Preneel, B. (2025). Impact of data duplication on deep neural network-based image classifiers: Robust vs. standard models. arXiv preprint arXiv:2504.00638.
Al Farizi, W. S., Hidayah, I., and Rizal, M. N. (2021). Isolation forest based anomaly detection: A systematic literature review. In 2021 8th International Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), pages 118–122. IEEE.
Gong, Y., Liu, G., Xue, Y., Li, R., and Meng, L. (2023). A survey on dataset quality in machine learning. Information and Software Technology, 162:107268.
Goyal, M., Lather, Y., and Lather, V. (2015). Analytical relation & comparison of psnr and ssim on babbon image and human eye perception using matlab. International Journal of Advanced Research in Engineering and Applied Sciences, 4(5):108–119.
Guo, L. L., Pfohl, S. R., Fries, J., Posada, J., Fleming, S. L., Aftandilian, C., Shah, N., and Sung, L. (2021). Systematic review of approaches to preserve machine learning performance in the presence of temporal dataset shift in clinical medicine. Applied clinical informatics, 12(04):808–815.
Habuza, T., Navaz, A. N., Hashim, F., Alnajjar, F., Zaki, N., Serhani, M. A., and Statsenko, Y. (2021). Ai applications in robotics, diagnostic image analysis and precision medicine: Current limitations, future trends, guidelines on cad systems for medicine. Informatics in Medicine Unlocked, 24:100596.
Kadhim, Y. A., Khan, M. U., and Mishra, A. (2022). Deep learning-based computer-aided diagnosis (cad): Applications for medical image datasets. Sensors, 22(22):8999.
Kim, T.-G. (2024). Hyperboliclr: Epoch insensitive learning rate scheduler. arXiv preprint arXiv:2407.15200.
Li, X., Li, T., Li, S., Tian, B., Ju, J., Liu, T., and Liu, H. (2023). Learning fusion feature representation for garbage image classification model in human–robot interaction. Infrared Physics & Technology, 128:104457.
Nassif, A. B., Talib, M. A., Nasir, Q., and Dakalbab, F. M. (2021). Machine learning for anomaly detection: A systematic review. Ieee Access, 9:78658–78700.
Rabasovic, M., Pavlovic, D., and Sevic, D. (2023). Analysis of laser ablation spectral data using dimensionality reduction techniques: Pca, t-sne and umap. Contrib. Astron. Obs. Skalnaté Pleso, 53(3):51–57.
Rivas, D. V., Pino, M. M., Pérez, Y. P., Rivas, S. V., Torres, O. G., Fernández, V. C., and Ysa, R. S. (2020). Comparison of distance methods for detection of atypical observations in monthly precipitation series. Revista de Climatología, 20.
Rodríguez-Rodríguez, J. A., López-Rubio, E., Ángel-Ruiz, J. A., and Molina-Cabello, M. A. (2024). The impact of noise and brightness on object detection methods. Sensors, 24(3):821.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022). Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278–25294.
Sharma, P., Ninomiya, T., Omodaka, K., Takahashi, N., Miya, T., Himori, N., Okatani, T., and Nakazawa, T. (2022). A lightweight deep learning model for automatic segmentation and analysis of ophthalmic images. Scientific reports, 12(1):8508.
Stewart, G. and Al-Khassaweneh, M. (2022). An implementation of the hdbscan* clustering algorithm. Applied Sciences, 12(5):2405.
Tan, J., Hou, B., Day, T., Simpson, J., Rueckert, D., and Kainz, B. (2021). Detecting outliers with poisson image interpolation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part V 24, pages 581–591. Springer.
Tan, M. and Le, Q. V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. CoRR, abs/1905.11946.
Thota, P. and Ramez, E. (2021). Web scraping of covid-19 news stories to create datasets for sentiment and emotion analysis. In Proceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference, pages 306–314.
Vieira, N. (2022). Accessibility in the legal amazon: Digital solutions. Climate Policy Initiative.
Wen, L., Gao, L., Li, X., and Zeng, B. (2021). Convolutional neural network with automatic learning rate scheduler for fault classification. IEEE Transactions on Instrumentation and Measurement, 70:1–12.
Xiao, K. and Ni, T. (2024). Computer-aided industrial product design based on image enhancement algorithm and convolutional neural network. Computer-Aided Design and Applications, 21(1).
Xu, Z., Liu, Z., Yan, Y., Liu, Z., Yu, G., and Xiong, C. (2024). Cleaner pretraining corpus curation with neural web scraping. arXiv preprint arXiv:2402.14652.
Zhang, Y., Zhan, Q., and Ma, Z. (2024). Efficientnet-eca: A lightweight network based on efficient channel attention for class-imbalanced welding defects classification. Advanced Engineering Informatics, 62:102737.
Publicado
19/07/2026
Como Citar
MOURA, Flávio et al.
VISHOD - Visual Dataset Scraping and Hybrid Outlier Detection. In: SEMINÁRIO INTEGRADO DE SOFTWARE E HARDWARE (SEMISH), 53. , 2026, Gramado/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 842-853.
ISSN 2595-6205.
DOI: https://doi.org/10.5753/semish.2026.19831.
