Text Recognition for Air Writing Systems: An Experimental Study

• Carlos E. S. Barbosa IFES
• Thiago B. Pereira IFES
• Israel M. do Carmo IFES
• Richard J. M. G. Tello IFES
• Francisco A. Boldt IFES
• Thiago M. Paixão IFES

DOI: https://doi.org/10.5753/eries.2024.244365

Abstract

This study explores Air Writing (AW) as a contactless human-machine interface for text input, assessing its feasibility with Optical Character Recognition (OCR). AW enables users to write in the air without physical surfaces, presenting challenges for recognition accuracy and user intent detection. Quantitative experiments using open-source OCR algorithms on simulated AW data demonstrate promising results, particularly with the implementation of stroke smoothing techniques. This research contributes valuable insights into improving AW’s practicality and OCR performance, aiming to enhance its usability across various interactive applications.

References

Abir, F. A., Siam, M. A., Sayeed, A., Hasan, M. A. M., and Shin, J. (2021). Deep learning based air-writing recognition with the choice of proper interpolation technique. Sensors, 21(24).

Bashir, M., Scharfenberg, G., and Kempf, J. (2011). Person authentication by handwriting in air using a biometric smart pen device. In BIOSIG 2011 – Proceedings of the Biometrics Special Interest Group, pages 219–226. Gesellschaft für Informatik e.V., Bonn.

Chen, M., AlRegib, G., and Juang, B.-H. (2016). Air-writing recognition—part i: Modeling and recognition of characters, words, and connecting motions. IEEE Transactions on Human-Machine Systems, 46(3):403–413.

Elshenaway, A. R. and Guirguis, S. K. (2021). On-air hand-drawn doodles for iot devices authentication during covid-19. IEEE Access, 9:161723–161744.

Itaguchi, Y., Yamada, C., and Fukuzawa, K. (2015). Writing in the air: Contributions of finger movement to cognitive processing. PLOS ONE, 10(6):1–17.

Lee, S.-K. and Kim, J.-H. (2021). Air-text: Air-writing and recognition system. In Proceedings of the 29th ACM International Conference on Multimedia, MM ’21, page 1267–1274, New York, NY, USA. Association for Computing Machinery.

Li, M., Lv, T., Cui, L., Lu, Y., Florencio, D., Zhang, C., Li, Z., and Wei, F. (2021). Trocr: Transformer-based optical character recognition with pre-trained models. [link].

Mukherjee, S., Ahmed, S. A., Dogra, D. P., Kar, S., and Roy, P. P. (2019). Fingertip detection and tracking for recognition of air-writing in videos. Expert Systems with Applications, 136:217–229.

Vaidya, V., Pravanth, T., and Viji, D. (2022). Air writing recognition application for dyslexic people. In 2022 International Mobile and Embedded Technology Conference (MECON), pages 553–558.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

Vloison, V. and Xiwei, H. (2021). Deep learning framework for line-level handwritten text recognition. [link].

Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.-L., and Grundmann, M. (2020). Mediapipe hands: On-device real-time hand tracking.