Real-Time Frequency Adjustment of Images and Videos

• Rafael L. Germano UFRGS
• Eduardo S. L. Gastal UFRGS

DOI: https://doi.org/10.5753/sibgrapi.est.2021.20012

Resumo

We present a technique for real-time adjustment of spatial frequencies in images and videos. Our method allows for both decreasing and increasing of frequencies, and is orthogonal to image resizing. Thus, it can be used to automatically adjust spatial frequencies to preserve the appearance of structured patterns during image downscaling and upscaling. By pre-computing the image's space-frequency decomposition and its unwrapped phases, these operations can be performed in real time, thanks to our novel mathematical perspective on frequency manipulation of digital images: interpreting the problem through the theory of instantaneous frequencies and phase unwrapping. To make this possible, we introduce an algorithm for the simultaneous phase unwrapping of several unordered frequency components, which also deals with the frequency-sign ambiguity of real signals. As such, our method provides theoretical and practical improvements to the concept of spectral remapping, enabling realtime performance and improved color handling. We demonstrate its effectiveness on video and a large number of images subject to frequency adjustment. By providing real-time control over the spatial frequencies associated with structured patterns, our technique expands the range of creative and technical possibilities for image and video processing.

Referências

S. G. Mallat, A Wavelet Tour of Signal Processing: The Sparse Way, 3rd ed. Elsevier, 2009.

E. S. L. Gastal and M. M. Oliveira, “Spectral remapping for image downscaling,” ACM Trans. Graph., vol. 36, no. 4, pp. 145:1–145:16, 2017.

D. Nehab and H. Hoppe, “A fresh look at generalized sampling,” Foundations and Trends in C.G. and Vision, vol. 8, no. 1, pp. 1–84, 2014.

A. V. Oppenheim and R. W. Schafer, Discrete-time signal processing. Pearson Higher Education, 2010.

D. Childers, D. Skinner, and R. Kemerait, “The cepstrum: A guide to processing,” Proc. of the IEEE, vol. 65, no. 10, pp. 1428–1443, 1977.

R. M. Goldstein, H. A. Zebker, and C. L. Werner, “Satellite radar interferometry: Two-dimensional phase unwrapping,” Radio Science, vol. 23, no. 4, pp. 713–720, Jul. 1988.

N. Delprat, B. Escudie, P. Guillemain, R. Kronland-Martinet, P. Tchamitchian, and B. Torresani, “Asymptotic wavelet and Gabor analysis: extraction of instantaneous frequencies,” IEEE Transactions on Information Theory, vol. 38, no. 2, pp. 644–664, Mar. 1992.

J. M. Lilly and S. C. Olhede, “On the Analytic Wavelet Transform,” IEEE Trans. on Information Theory, vol. 56, no. 8, pp. 4135–4156, Aug. 2010.

P. Guillemain and R. Kronland-Martinet, “Characterization of acoustic signals through continuous linear time-frequency representations,” Proceedings of the IEEE, vol. 84, no. 4, pp. 561–585, Apr. 1996.

D. C. Ghiglia and M. D. Pritt, Two-dimensional phase unwrapping: theory, algorithms, and software. Wiley, Apr. 1998.

L. Sacht and D. Nehab, “Optimized Quasi-Interpolators for Image Reconstruction,” IEEE TIP, vol. 24, no. 12, pp. 5249–5259, 2015.

L. N. Mertz, “Speckle imaging, photon by photon,” Applied Optics, vol. 18, no. 5, pp. 611–614, Mar. 1979.

J. Solomon, Numerical algorithms: methods for computer vision, machine learning, and graphics. CRC Press, Taylor & Francis, 2015.

E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Computer Graphics and Applications, vol. 21, no. 5, pp. 34–41, Jul. 2001.

T. A. Davis, “Algorithm 915, SuiteSparseQR: Multifrontal Multithreaded Rank-revealing Sparse QR Factorization,” vol. 38, no. 1, pp. 8:1–8:22, 2011.

R. L. Germano, M. M. Oliveira, and E. S. L. Gastal, “Real-time frequency adjustment of images and videos,” Computer Graphics Forum, vol. 40, no. 2, pp. 23–37, 2021. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.142612