Exploring Double Cross Cyclic Interpolation in Unpaired Image-to-Image Translation
Resumo
The unpaired image-to-image translation consists of transferring a sample a in the domain A to an analog sample b in the domain B without intensive pixel-to-pixel supervision. The current vision focuses on learning a generative function that maps both domains but ignoring the latent information, although its exploration is not explicit supervision. This paper proposes a cross-domain GAN-based model to achieve a bi-directional translation guided by latent space supervision. The proposed architecture provides a double-loop cyclic reconstruction loss in an exchangeable training adopted to reduce mode collapse and enhance local details. Our proposal has outstanding results in visual quality, stability, and pixel-level segmentation metrics over different public datasets.
Palavras-chave:
Cross domain interpolation, Unpaired Image to Image Translation, Latent space exploration
Referências
P. Isola J.-Y. Zhu T. Zhou A. A. Efros "Image-to-image translation with conditional adversarial networks" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 1125-12017.
S. U. Dar M. Yurt L. Karacan A. Erdem E. Erdem T. Cukur "Image synthesis in multi-contrast mri with conditional generative adversarial networks" IEEE transactions on medical imaging 2019.
A. Mauricio J. Lopez R. Huauya J. Diaz "High-resolution generative adversarial neural networks applied to histological images generation" International Conference on Artificial Neural Networks pp. 195-2018.
Y. Wang X. Tao X. Qi X. Shen J. Jia "Image inpainting via generative multi-column convolutional neural networks" Advances in Neural Information Processing Systems pp. 331-2018.
W. Wang Q. Huang S. You C. Yang U. Neumann "Shape inpainting using 3d generative adversarial network and recurrent convolutional networks" Proceedings of the IEEE International Conference on Computer Vision pp. 2298-22017.
P. Luc C. Couprie S. Chintala J. Verbeek Semantic segmentation using adversarial networks 2016.
J. Long E. Shelhamer T. Darrell "Fully convolutional networks for semantic segmentation" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 3431-32015.
R. Zhang T. Pfister J. Li Harmonic unpaired image-to-image translation 2019.
J.-Y. Zhu T. Park P. Isola A. A. Efros "Unpaired image-to-image translation using cycle-consistent adversarial networks" Proceedings of the IEEE international conference on computer vision pp. 2223-22017.
X. Huang M.-Y. Liu S. Belongie J. Kautz "Multimodal unsupervised image-to-image translation" Proceedings of the European Conference on Computer Vision (ECCV) pp. 172-2018.
S. Benaim L. Wolf "One-shot unsupervised cross domain translation" Advances in Neural Information Processing Systems pp. 2104-22018.
L. Chen H. Zhang J. Xiao W. Liu S.-F. Chang "Zero-shot visual recognition using semantics-preserving adversarial embedding networks" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 1043-1052 2018.
J.-Y. Zhu P. Krahenbuhl E. Shechtman A. A. Efros "Generative visual manipulation on the natural image manifold" European Conference on Computer Vision pp. 597-2016.
M. Li H. Huang L. Ma W. Liu T. Zhang Y. Jiang "Unsupervised image-to-image translation with stacked cycle-consistent adversarial networks" Proceedings of the European Conference on Computer Vision (ECCV) pp. 184-2018.
Z. Yi H. Zhang P. Tan M. Gong "Dualgan: Unsupervised dual learning for image-to-image translation" Proceedings of the IEEE international conference on computer vision pp. 2849-22017.
J. Lopez A. Mauricio J. Diaz C. Guillermo "Cross-domain interpolation for unpaired image-to-image translation" International Conference on Computer Vision Systems pp. 120-2019.
A. Hertzmann C. E. Jacobs N. Oliver B. Curless D. H. Salesin "Image analogies" Proceedings of the 28th annual conference on Computer graphics and interactive techniques pp. 327-2001.
A. A. Efros W. T. Freeman "Image quilting for texture synthesis and transfer" Proceedings of the 28th annual conference on Computer graphics and interactive techniques pp. 341-2001.
L. Gatys A. S. Ecker M. Bethge "Texture synthesis using convolutional neural networks" Advances in neural information processing systems pp. 262-2015.
L. A. Gatys A. S. Ecker M. Bethge "Image style transfer using convolutional neural networks" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 2414-22016.
A. Shrivastava T. Pfister O. Tuzel J. Susskind W. Wang R. Webb "Learning from simulated and unsupervised images through adversarial training" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 2107-22017.
I. Goodfellow J. Pouget-Abadie M. Mirza B. Xu D. Warde-Farley S. Ozair A. Courville Y. Bengio "Generative adversarial nets" Advances in neural information processing systems pp. 2672-22014.
Y. Taigman A. Polyak L. Wolf Unsupervised cross-domain image generation 2016.
D. He Y. Xia T. Qin L. Wang N. Yu T.-Y. Liu W.-Y. Ma "Dual learning for machine translation" Advances in Neural Information Processing Systems pp. 820-2016.
J.-Y. Zhu R. Zhang D. Pathak T. Darrell A. A. Efros O. Wang E. Shechtman "Toward multimodal image-to-image translation" Advances in Neural Information Processing Systems pp. 465-2017.
Y. Hiasa Y. Otake M. Takao T. Matsuoka K. Takashima A. Carass J. L. Prince N. Sugano Y. Sato "Cross-modality image synthesis from unpaired data using cyclegan" International Workshop on Simulation and Synthesis in Medical Imaging pp. 31-41 2018.
H.-Y. Lee H.-Y. Tseng J.-B. Huang M. Singh M.-H. Yang "Diverse image-to-image translation via disentangled representations" Proceedings of the European Conference on Computer Vision (ECCV) pp. 35-51 2018.
A. Gonzalez-Garcia J. van de Weijer Y. Bengio "Image-to-image translation for cross-domain disentanglement" Advances in Neural Information Processing Systems pp. 1287-12018.
X. Mao Q. Li H. Xie R. Y. Lau Z. Wang S. Paul Smolley "Least squares generative adversarial networks" Proceedings of the IEEE International Conference on Computer Vision pp. 2794-22017.
M. Arjovsky S. Chintala L. Bottou Wasserstein gan 2017.
I. Gulrajani F. Ahmed M. Arjovsky V. Dumoulin A. C. Courville "Improved training of wasserstein gans" Advances in Neural Information Processing Systems pp. 5767-52017.
S. U. Dar M. Yurt L. Karacan A. Erdem E. Erdem T. Cukur "Image synthesis in multi-contrast mri with conditional generative adversarial networks" IEEE transactions on medical imaging 2019.
A. Mauricio J. Lopez R. Huauya J. Diaz "High-resolution generative adversarial neural networks applied to histological images generation" International Conference on Artificial Neural Networks pp. 195-2018.
Y. Wang X. Tao X. Qi X. Shen J. Jia "Image inpainting via generative multi-column convolutional neural networks" Advances in Neural Information Processing Systems pp. 331-2018.
W. Wang Q. Huang S. You C. Yang U. Neumann "Shape inpainting using 3d generative adversarial network and recurrent convolutional networks" Proceedings of the IEEE International Conference on Computer Vision pp. 2298-22017.
P. Luc C. Couprie S. Chintala J. Verbeek Semantic segmentation using adversarial networks 2016.
J. Long E. Shelhamer T. Darrell "Fully convolutional networks for semantic segmentation" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 3431-32015.
R. Zhang T. Pfister J. Li Harmonic unpaired image-to-image translation 2019.
J.-Y. Zhu T. Park P. Isola A. A. Efros "Unpaired image-to-image translation using cycle-consistent adversarial networks" Proceedings of the IEEE international conference on computer vision pp. 2223-22017.
X. Huang M.-Y. Liu S. Belongie J. Kautz "Multimodal unsupervised image-to-image translation" Proceedings of the European Conference on Computer Vision (ECCV) pp. 172-2018.
S. Benaim L. Wolf "One-shot unsupervised cross domain translation" Advances in Neural Information Processing Systems pp. 2104-22018.
L. Chen H. Zhang J. Xiao W. Liu S.-F. Chang "Zero-shot visual recognition using semantics-preserving adversarial embedding networks" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 1043-1052 2018.
J.-Y. Zhu P. Krahenbuhl E. Shechtman A. A. Efros "Generative visual manipulation on the natural image manifold" European Conference on Computer Vision pp. 597-2016.
M. Li H. Huang L. Ma W. Liu T. Zhang Y. Jiang "Unsupervised image-to-image translation with stacked cycle-consistent adversarial networks" Proceedings of the European Conference on Computer Vision (ECCV) pp. 184-2018.
Z. Yi H. Zhang P. Tan M. Gong "Dualgan: Unsupervised dual learning for image-to-image translation" Proceedings of the IEEE international conference on computer vision pp. 2849-22017.
J. Lopez A. Mauricio J. Diaz C. Guillermo "Cross-domain interpolation for unpaired image-to-image translation" International Conference on Computer Vision Systems pp. 120-2019.
A. Hertzmann C. E. Jacobs N. Oliver B. Curless D. H. Salesin "Image analogies" Proceedings of the 28th annual conference on Computer graphics and interactive techniques pp. 327-2001.
A. A. Efros W. T. Freeman "Image quilting for texture synthesis and transfer" Proceedings of the 28th annual conference on Computer graphics and interactive techniques pp. 341-2001.
L. Gatys A. S. Ecker M. Bethge "Texture synthesis using convolutional neural networks" Advances in neural information processing systems pp. 262-2015.
L. A. Gatys A. S. Ecker M. Bethge "Image style transfer using convolutional neural networks" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 2414-22016.
A. Shrivastava T. Pfister O. Tuzel J. Susskind W. Wang R. Webb "Learning from simulated and unsupervised images through adversarial training" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 2107-22017.
I. Goodfellow J. Pouget-Abadie M. Mirza B. Xu D. Warde-Farley S. Ozair A. Courville Y. Bengio "Generative adversarial nets" Advances in neural information processing systems pp. 2672-22014.
Y. Taigman A. Polyak L. Wolf Unsupervised cross-domain image generation 2016.
D. He Y. Xia T. Qin L. Wang N. Yu T.-Y. Liu W.-Y. Ma "Dual learning for machine translation" Advances in Neural Information Processing Systems pp. 820-2016.
J.-Y. Zhu R. Zhang D. Pathak T. Darrell A. A. Efros O. Wang E. Shechtman "Toward multimodal image-to-image translation" Advances in Neural Information Processing Systems pp. 465-2017.
Y. Hiasa Y. Otake M. Takao T. Matsuoka K. Takashima A. Carass J. L. Prince N. Sugano Y. Sato "Cross-modality image synthesis from unpaired data using cyclegan" International Workshop on Simulation and Synthesis in Medical Imaging pp. 31-41 2018.
H.-Y. Lee H.-Y. Tseng J.-B. Huang M. Singh M.-H. Yang "Diverse image-to-image translation via disentangled representations" Proceedings of the European Conference on Computer Vision (ECCV) pp. 35-51 2018.
A. Gonzalez-Garcia J. van de Weijer Y. Bengio "Image-to-image translation for cross-domain disentanglement" Advances in Neural Information Processing Systems pp. 1287-12018.
X. Mao Q. Li H. Xie R. Y. Lau Z. Wang S. Paul Smolley "Least squares generative adversarial networks" Proceedings of the IEEE International Conference on Computer Vision pp. 2794-22017.
M. Arjovsky S. Chintala L. Bottou Wasserstein gan 2017.
I. Gulrajani F. Ahmed M. Arjovsky V. Dumoulin A. C. Courville "Improved training of wasserstein gans" Advances in Neural Information Processing Systems pp. 5767-52017.
Publicado
28/10/2019
Como Citar
CÁCERES, Jorge Roberto López ; MAURICIO, Manasses A. ; CÁMARA, Guillermo.
Exploring Double Cross Cyclic Interpolation in Unpaired Image-to-Image Translation. In: CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 32. , 2019, Rio de Janeiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
DOI: https://doi.org/10.5753/sibgrapi.2019.9798.
