Animation Frame Colorization with GANs
Resumo
This research paper presents an innovative approach to alleviate the labor-intensive nature of traditional 2D handmade animation utilizing artificial intelligence techniques. Specifically, we focus on refining the process of image colorization for 2D animations by employing Generative Adversarial Networks (GANs). The proposed method involves leveraging the power of GANs to paint a sequence of black and white frames in a manner that emulates the colors present in a single colored example.
Referências
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” 2014. [Online]. Available: [link]
C. Furusawa, K. Hiroshiba, K. Ogaki, and Y. Odagiri, “Comicolorization: Semi-automatic manga colorization,” 2017. [Online]. Available: [link]
Y. Qu, T.-T. Wong, and P.-A. Heng, “Manga colorization,” ACM Transactions on Graphics (ToG), vol. 25, no. 3, pp. 1214–1220, 2006.
H. Thasarathan, K. Nazeri, and M. Ebrahimi, “Automatic temporally coherent video colorization,” in 2019 16th conference on computer and robot vision (CRV). IEEE, 2019, pp. 189–194.
M. Shi, J.-Q. Zhang, S.-Y. Chen, L. Gao, Y.-K. Lai, and F.-L. Zhang, “Deep line art video colorization with a few references,” arXiv preprint arXiv:2003.10685, 2020.
R. Nascimento, F. Queiroz, A. Rocha, T. Ing Ren, V. Mello, and A. Peixoto, “Computer-assisted coloring and illuminating based on a region-tree structure,” SpringerPlus, vol. 1, p. 1, 03 2012.
K. Sato, Y. Matsui, T. Yamasaki, and K. Aizawa, “Reference-based manga colorization by graph correspondence using quadratic programming,” in SIGGRAPH Asia 2014 Technical Briefs, 2014, pp. 1–4.
A. Odena, V. Dumoulin, and C. Olah, “Deconvolution and checkerboard artifacts,” Distill, 2016. [Online]. Available: [link]
J. Canny, “A computational approach to edge detection,” IEEE Transactions on pattern analysis and machine intelligence, no. 6, pp. 679–698, 1986.
J. Bescós, G. Cisneros, J. M. Martínez, J. M. Menéndez, and J. Cabrera, “A unified model for techniques on video-shot transition detection,” IEEE transactions on multimedia, vol. 7, no. 2, pp. 293–307, 2005.
T. Souček and J. Lokoč, “Transnet v2: An effective deep network architecture for fast shot transition detection,” arXiv preprint arXiv:2008.04838, 2020.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
F. A. Fardo, V. H. Conforto, F. C. de Oliveira, and P. S. Rodrigues, “A formal evaluation of psnr as quality measurement parameter for image segmentation algorithms,” arXiv preprint arXiv:1605.07116, 2016.
Y. Benny, T. Galanti, S. Benaim, and L. Wolf, “Evaluation metrics for conditional image generation,” International Journal of Computer Vision, vol. 129, pp. 1712–1731, 2021.
J. Nilsson and T. Akenine-Möller, “Understanding ssim,” arXiv preprint arXiv:2006.13846, 2020.
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” 2017.
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 6840–6851. [Online]. Available: [link].
Z. Wang, H. Zheng, P. He, W. Chen, and M. Zhou, “Diffusion-gan: Training gans with diffusion,” arXiv preprint arXiv:2206.02262, 2022.
C. Furusawa, K. Hiroshiba, K. Ogaki, and Y. Odagiri, “Comicolorization: Semi-automatic manga colorization,” 2017. [Online]. Available: [link]
Y. Qu, T.-T. Wong, and P.-A. Heng, “Manga colorization,” ACM Transactions on Graphics (ToG), vol. 25, no. 3, pp. 1214–1220, 2006.
H. Thasarathan, K. Nazeri, and M. Ebrahimi, “Automatic temporally coherent video colorization,” in 2019 16th conference on computer and robot vision (CRV). IEEE, 2019, pp. 189–194.
M. Shi, J.-Q. Zhang, S.-Y. Chen, L. Gao, Y.-K. Lai, and F.-L. Zhang, “Deep line art video colorization with a few references,” arXiv preprint arXiv:2003.10685, 2020.
R. Nascimento, F. Queiroz, A. Rocha, T. Ing Ren, V. Mello, and A. Peixoto, “Computer-assisted coloring and illuminating based on a region-tree structure,” SpringerPlus, vol. 1, p. 1, 03 2012.
K. Sato, Y. Matsui, T. Yamasaki, and K. Aizawa, “Reference-based manga colorization by graph correspondence using quadratic programming,” in SIGGRAPH Asia 2014 Technical Briefs, 2014, pp. 1–4.
A. Odena, V. Dumoulin, and C. Olah, “Deconvolution and checkerboard artifacts,” Distill, 2016. [Online]. Available: [link]
J. Canny, “A computational approach to edge detection,” IEEE Transactions on pattern analysis and machine intelligence, no. 6, pp. 679–698, 1986.
J. Bescós, G. Cisneros, J. M. Martínez, J. M. Menéndez, and J. Cabrera, “A unified model for techniques on video-shot transition detection,” IEEE transactions on multimedia, vol. 7, no. 2, pp. 293–307, 2005.
T. Souček and J. Lokoč, “Transnet v2: An effective deep network architecture for fast shot transition detection,” arXiv preprint arXiv:2008.04838, 2020.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
F. A. Fardo, V. H. Conforto, F. C. de Oliveira, and P. S. Rodrigues, “A formal evaluation of psnr as quality measurement parameter for image segmentation algorithms,” arXiv preprint arXiv:1605.07116, 2016.
Y. Benny, T. Galanti, S. Benaim, and L. Wolf, “Evaluation metrics for conditional image generation,” International Journal of Computer Vision, vol. 129, pp. 1712–1731, 2021.
J. Nilsson and T. Akenine-Möller, “Understanding ssim,” arXiv preprint arXiv:2006.13846, 2020.
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” 2017.
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 6840–6851. [Online]. Available: [link].
Z. Wang, H. Zheng, P. He, W. Chen, and M. Zhou, “Diffusion-gan: Training gans with diffusion,” arXiv preprint arXiv:2206.02262, 2022.
Publicado
06/11/2023
Como Citar
NOGUEIRA, João Vitor Santiago; CARVALHO, Leonardo de Oliveira.
Animation Frame Colorization with GANs. In: WORKSHOP DE TRABALHOS DA GRADUAÇÃO - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 36. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 132-135.
DOI: https://doi.org/10.5753/sibgrapi.est.2023.27465.
