Subjective Metrics for Usability Evaluation of Automatic Transcription Systems in Radiology Reports

• Valéria Farinazzo Martins UPM
• Lincoln de Assis Moura Jr. Assis Moura eHealth

Abstract

This article describes an usability evaluation for Automatic Transcription System of Radiology Reports through subjective metrics proposed and applied to a case study, using a plan of usability tests in order to validate the process. This review aimed to be considered secondary low cost because they were used, first, volunteers and users only after the metrics were applied to the system worked in the hospital, with real users. This is important because it reduces the time that radiologists have a limitation in their activities.

References

Alapetite, A., Boje, A.H. and Morten, H. (2009) Acceptance of speech recognition by physicians : a survey of expectations, experiences, and social influence, In: International journal of human-computer studies, ISSN 1071-5819, v. 67, n. 1, p. 36-49.

Bernsen, N.O. and Dybkjaer, L. (2001) Usability evaluation in spoken language dialogue systems and their components. In: Proceedings of the workshop on Evaluation for Language and Dialogue Systems, v. 9, Toulouse, France.

Bhan, S.N., Coblentz, C., Norman, G.R. and Ali, A.H. (2008) Effect of voice recognition on radiologist reporting time. In: CARJ, v. 59, n. 4, October.

Chin, J. P., Diehl, V. A. and Norman, K. L. (1998) Development of an instrument measuring user satisfaction of the human-computer interface. In: ACM CHI'88 Proceedings, p. 213-218.

Durling, S. and Lumsden, J. (2008) Speech Recognition Use in Healthcare Applications. In: Proceedings of MoMM2008, MoCoHe, ACM 978-1-60558-269-6/08/0011.

Dybkjaer, L. and Bernsen, N.O. (2001) Usability evaluation in spoken language dialogue systems. In: Proceedings of the ACL 2001 Workshop on Evaluation Methodologies for Language and Dialogue Systems.

Hartikainen, M., Salonen, E. and Turunen, M. (2004) Subjective evaluation of spoken dialogue systems using SER VQUAL method. In: INTERSPEECH-2004, p.2273-2276.

Hundt, W. et al. (1999) Speech processing in radiology. In: Eur Radiol n.9, p. 1451-1456.

Larsen, L.B. Issues in the evaluation of spoken dialogue systems using objective and subjective measures. In: Proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU'03), St. Thomas, U.S. Virgin Islands, USA, 2003.

Mohr, D.N. et al. (2003) Speech recognition as a transcription AID: A randomized comparison with standard transcription. In: Journal of Medical Informatics, v. 10, n. 1, p. 85-93.

Möller, S. (2005) Quality of telephone-based spoken dialogue systems, New York, NY: Springer.

Nuance, Disponível em: [link]. Acesso em 20/12/2010.

Robbins, A.H. et al. (1987) Speech-controlled generation of radiology reports. In: Radiology n. 164, p. 569-573.

Rodger, J. A. and Pendharkar, P. C. (2007) A field study of database communication issues peculiar to users of a voice activated medical tracking application. In: Decision Support Systems, v. 43, n. 1, p. 168-180.

Snomed. Disponível em: [link]. Acesso em 07/03/2012.

Walker, M. A., Passnneau, R. and Boland, J. E. Quantitative and qualitative evaluation of Darpa communicator spoken dialogue systems. In: Proceedings of the 39rd Annual Meeting on Association for Computational Linguistics, Toulouse, France, 2001.

White, K. S. (2005) Speech recognition implementation in radiology, New York: Springer-Verlag, 10.1007/s00247-005-1511-x.