Open Set Domain Adaptation Methods in Deep Networks for Image Recognition

• Lucas Fernando Alvarenga e Silva UNICAMP
• Jurandy Almeida UFSCar

DOI: https://doi.org/10.5753/sibgrapi.est.2023.27454

Resumo

Deep learning (DL) has revolutionized various fields through its remarkable capacity to learn from raw data. However, in uncontrolled environments like in the wild, the performance of these systems might degrade to some extent, especially with unlabeled datasets. Naive approaches train DL models on labeled datasets (source domains) that resemble the unlabeled test dataset (target domain), but nonetheless, this approach may not yield optimal results due to domain and category-shift problems. These issues have been the primary focus of Unsupervised Domain Adaptation (UDA) and Open Set Recognition research areas. To address the domain-shift problem, we introduced the Multi-Source Domain Alignment Layers (MS-DIAL), a structural solution for multi-source UDA. MS-DIAL aligns the source domains and the target domain at various levels of the feature space, individually achieving competitive results comparable to the state-of-the-art, and when combined with other UDA methods, it further enhances transferability by up to 30.64% in relative performance gains. Subsequently, we tackled the demanding setup of Open Set Domain Adaptation (OSDA), where both domain and category-shift issues coexist. Our proposed approach involves dealing with negatives, extracting a high-confidence set of unknown instances, and using them as a hard constraint to refine the classification boundaries of OSDA methods. We assessed our proposal in an extensive set of experiments, which achieved up to 5.8% of absolute performance gains.

Referências

C. Geng, S. Huang, and S. Chen, “Recent advances in open set recognition: A survey,” TPAMI, vol. 43, no. 10, pp. 3614–3631, 2021.

S. Bucci, M. R. Loghmani, and T. Tommasi, “On the effectiveness of image rotation for open set domain adaptation,” in ECCV, 2020, pp. 422–438.

C. Saltori, P. Rota, N. Sebe, and J. Almeida, “Low-budget label query through domain alignment enforcement,” Computer Vision and Image Understanding, vol. 222, p. 103485, 2022.

L. F. A. Silva, D. C. G. Pedronette, F. A. Faria, J. P. Papa, and J. Almeida, “Improving transferability of domain adaptation networks through domain alignment layers,” in SIBGRAPI, 2021, pp. 168—-175.

P. P. Busto and J. Gall, “Open set domain adaptation,” in ICCV, 2017, pp. 754–763.

Y. Xu and D. Klabjan, “Open set domain adaptation by extreme value theory,” CoRR, vol. abs/2101.02561, 2021.

S. Bucci, F. C. Borlino, B. Caputo, and T. Tommasi, “Distancebased hyperspherical classification for multi-source open-set domain adaptation,” in WACV, 2022, pp. 1030–1039.

K. Saito, S. Yamamoto, Y. Ushiku, and T. Harada, “Open set domain adaptation by backpropagation,” in ECCV, 2018, pp. 156–171.

S. Rakshit, D. Tamboli, P. S. Meshram, B. Banerjee, G. Roig, and S. Chaudhuri, “Multi-source open-set deep adversarial domain adaptation,” in ECCV, 2020, pp. 735–750.

F. M. Carlucci, L. Porzi, B. Caputo, E. Ricci, and S. R. Bulò, “Just DIAL: domain alignment layers for unsupervised domain adaptation,” in ICIAP, 2017, pp. 357–369.

X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, “Moment matching for multi-source domain adaptation,” in ICCV, 2019, pp. 1406–1415.

R. Xu, Z. Chen, W. Zuo, J. Yan, and L. Lin, “Deep cocktail network: Multi-source unsupervised domain adaptation with category shift,” in CVPR, 2018, pp. 3964–3973.

J. Wen, R. Greiner, and D. Schuurmans, “Domain aggregation networks for multi-source domain adaptation,” in ICML, 2020, pp. 10 927–10 937.

S. Zhao, G. Wang, S. Zhang, Y. Gu, Y. Li, Z. Song, P. Xu, R. Hu, H. Chai, and K. Keutzer, “Multi-source distilling domain adaptation,” in AAAI, 2020, pp. 12 975–12 983.

J. Liu, X. Guo, and Y. Yuan, “Unknown-oriented learning for open set domain adaptation,” in ECCV, 2022, pp. 334–350.

M. Baktashmotlagh, T. Chen, and M. Salzmann, “Learning to generate the unknowns as a remedy to the open-set domain shift,” in WACV, 2022, pp. 3737–3746.

Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. S. Lempitsky, “Domain-adversarial training of neural networks,” JMLR, vol. 17, pp. 59:1–59:35, 2016.

H. Yan, Y. Ding, P. Li, Q. Wang, Y. Xu, and W. Zuo, “Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation,” in CVPR, 2017, pp. 945–954.

F. M. Carlucci, L. Porzi, B. Caputo, E. Ricci, and S. R. Bulò, “Autodial: Automatic domain alignment layers,” in ICCV, 2017, pp. 5077–5085.

Y. Li, M. Murias, G. Dawson, and D. E. Carlson, “Extracting relationships by multi-domain matching,” in NeurIPS, 2018, pp. 6799–6810.

H. Zhao, S. Zhang, G. Wu, J. M. F. Moura, J. P. Costeira, and G. J. Gordon, “Adversarial multiple source domain adaptation,” in NeurIPS, 2018, pp. 8568–8579.

L. Neal, M. Olson, X. Fern, W.-K. Wong, and F. Li, “Open set learning with counterfactual images,” in ECCV, 2018, pp. 620–635.

G. Chen, P. Peng, X. Wang, and Y. Tian, “Adversarial reciprocal points learning for open set recognition,” TPAMI, vol. 44, no. 11, pp. 8065–8081, 2022.

S. Vaze, K. Han, A. Vedaldi, and A. Zisserman, “Open-set recognition: A good closed-set classifier is all you need,” in ICLR, 2022.

L. F. A. Silva and J. Almeida, “Ms-dial: Multi-source domain alignment layers for unsupervised domain adaptation,” in WVC, 2020, pp. 111–116.

K. Saito, D. Kim, S. Sclaroff, and K. Saenko, “Universal domain adaptation through self-supervision,” in NeurIPS, 2020.

K. Saito and K. Saenko, “Ovanet: One-vs-all network for universal domain adaptation,” in ICCV, 2021, pp. 9000–9009.

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in ICML, 2015, pp. 448–456.

K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual category models to new domains,” in ECCV, 2010, pp. 213–226.

H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,” in CVPR, 2017, pp. 5385–5394.

G. French, M. Mackiewicz, and M. Fisher, “Self-ensembling for visual domain adaptation,” in ICLR, 2018.

L. F. A. Silva, N. Sebe, and J. Almeida, “Tightening classification boundaries in open set domain adaptation through unknown exploitation,” in SIBGRAPI, 2023, pp. 1–6.

A. Agostinelli, J. R. R. Uijlings, T. Mensink, and V. Ferrari, “Transferability metrics for selecting source model ensembles,” in CVPR, 2022, pp. 7926–7936.

Z. Wang, Q. She, and T. E. Ward, “Generative adversarial networks in computer vision: A survey and taxonomy,” ACS, vol. 54, no. 2, pp. 37:1–37:38, 2022.

G. C. Rocha, H. M. Paiva, D. G. Sanches, D. Fiks, R. M. Castro, and L. F. A. Silva, “Information system for epidemic control: a computational solution addressing successful experiences and main challenges,” Library Hi Tech, vol. 39, no. 3, pp. 834–854, 2021.

M. Miranda, L. F. A. e Silva, S. Santos, V. S. Jr, T. Körting, and J. Almeida, “A high-spatial resolution dataset and few-shot deep learning benchmark for image classification,” in SIBGRAPI, 2022, pp. 19–24.