Acceptance and Usability of Complex Medical Systems: A Study with Radiology Professionals
Resumo
The increasing demand for imaging tests has made radiology information systems crucial in medical practice, especially those based on web technology. These systems include Picture Archiving and Communication System (PACS), Radiology Information Systems (RIS), and Hospital Information System (HIS), generate and manipulate images through specialized software. To operate this complex software, require attention to detail and image manipulation techniques for accurate diagnoses. Usability issues in medical image manipulation software, given the process of adapting to new software and complex tasks, can result in inaccurate diagnoses with clinical impact. This is a qualitative study, which is based on the work routines of radiology professionals, focusing on issues of cognitive learning, interaction, and usability with radiology software. Moderate usability tests with radiology technicians were conducted to identify the difficulties and challenges they encounter while using medical image manipulation software. The analysis identified 64 problems grouped into 20 categories and organized under Visual Presentation, Content, Information Architecture, and Interactivity. The paper emphasizes violated heuristics and describes how these problem categories impact users in their medical activities and their influence on the clinical process. The obtained results provide insights to enhance usability practices and recommendations, aiming to support the development systems used in radiology practice.
Palavras-chave:
usability issues, radiology systems, qualitative analysis
Referências
Reza Abbasi, Monireh Sadeqi Jabali, Reza Khajouei, and Hamidreza Tadayon. 2020. Investigating the satisfaction level of physicians in regards to implementing medical Picture Archiving and Communication System (PACS). BMC medical informatics and decision making 20, 1 (2020), 1–8.
M. Alhajeri and S.G.S. Shah. 2019. Limitations in and Solutions for Improving the Functionality of Picture Archiving and Communication System: an Exploratory Study of PACS Professionals’ Perspectives. Journal of Digital Imaging 32 (2019), 54–67.
Priya Darshini B, Deepan Chakkaravarthy N, Gokul B, and Sabari Maharaja B. 2023. A Web-based Dicom Image and Plane Viewer. In 2023 Fifth International Conference on Electrical, Computer and Communication Technologies (ICECCT). IEEE, 1–7.
Rade R Babić, Zoran Milošević, and Gordana Stanković-Babić. 2012. Web technology in health information system. Acta Facultatis Medicae Naissensis 29, 2 (2012), 81–87.
Mirza Mansoor Baig, Hamid GholamHosseini, Aasia A Moqeem, Farhaan Mirza, and Maria Lindén. 2017. A systematic review of wearable patient monitoring systems–current challenges and opportunities for clinical adoption. Journal of medical systems 41, 7 (2017), 1–9.
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
Cronin, B. Kane, and G. Doherty. 2021. A Qualitative Analysis of the Needs and Experiences of Hospital-based Clinicians when Accessing Medical Imaging. Journal of digital imaging (2021), 1––12. DOI: 10.1007/s10278-021-00446-1
Fábio A. C. da Silva, André P. Freire, and Marluce R. Pereira. 2022. Understanding Interaction and Organizational Issues in Radiology Information Systems: a Qualitative Study with Health Professionals. In XVIII Brazilian Symposium on Information Systems. –.
Camila Rodrigues Dias, Marluce Rodrigues Pereira, and Andre Pimenta Freire. 2017. Qualitative review of usability problems in health information systems for radiology. Journal of biomedical informatics 76 (2017), 19–33.
K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. MIT Press, Cambridge, MA, USA.
Misagh Zahiri Esfahani, Reza Khajouei, and Mohammad Reza Baneshi. 2018. Augmentation of the think aloud method with users’ perspectives for the selection of a picture archiving and communication system. Journal of Biomedical Informatics 80 (2018), 43–51.
Ina Geldermann, Christoph Grouls, Christiane Kuhl, Thomas M Deserno, and Cord Spreckelsen. 2013. Black box integration of computer-aided diagnosis into PACS deserves a second chance: results of a usability study concerning bone age assessment. Journal of Digital Imaging 6 (2013), 698—-708.
Hai K Huang. 2003. Enterprise PACS and image distribution. Computerized Medical Imaging and Graphics 27, 2-3 (2003), 241–253.
Fateme Rangraz Jeddi, Ehsan Nabovati, Reyhane Bigham, and Razieh Farrahi. 2020. Usability evaluation of a comprehensive national health information system: A heuristic evaluation. Informatics in Medicine Unlocked 19 (2020), 100332.
Wiard Jorritsma, Fokie Cnossen, Rudi A Dierckx, Matthijs Oudkerk, and Peter MA van Ooijen. 2016. Pattern mining of user interaction logs for a post-deployment usability evaluation of a radiology PACS client. International Journal of Medical Informatics 85, 1 (2016), 36–42.
Wiard Jorritsma, Fokie Cnossen, Rudi A Dierckx, Matthijs Oudkerk, and Peter MA Van Ooijen. 2016. Post-deployment usability evaluation of a radiology workstation. International Journal of Medical Informatics 85, 1 (2016), 28–35.
Wiard Jorritsma, Fokie Cnossen, and Peter MA van Ooijen. 2014. Merits of usability testing for PACS selection. International Journal of Medical Informatics 83, 1 (2014), 27–36.
Wiard Jorritsma, Fokie Cnossen, and Peter MA van Ooijen. 2015. Adaptive support for user interface customization: a study in radiology. International Journal of Human-Computer Studies 77 (2015), 1–9.
Brandan Kennedy, Ellen Kerns, Y Raymond Chan, Barbara S Chaparro, and Sarah D Fouquet. 2019. Safeuristics! Do Heuristic Evaluation Violation Severity Ratings Correlate with Patient Safety Severity Ratings for a Native Electronic Health Record Mobile Application? Applied clinical informatics 10, 02 (2019), 210–218.
Dasueran Kim, Peter Kang, Jungmin Yun, Sung-Hye Park, Jeong-Wook Seo, and Peom Park. 2014. Study on User Interface of Pathology Picture Archiving and Communication System. Healthcare Informatics Research 20, 1 (2014), 45–51.
Andre W Kushniruk and Vimla L Patel. 2004. Cognitive and usability engineering methods for the evaluation of clinical information systems. Journal of biomedical informatics 37, 1 (2004), 56–76.
J. Lazar, J. H. Feng, and H. Hochheiser. 2017. Research Methods in Human-Computer Interaction. Morgan Kaufmann is an imprint of Elsevier, Cambridge, MA, USA.
James R Lewis. 1995. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7, 1 (1995), 57–78.
Svetlana Z Lowry, Patricia Abbott, Michael C Gibbons, Svetlana Z Lowry, Robert North, Emily S Patterson, Matthew T Quinn, Mala Ramaiah, Robert M Schumacher, and Jiajie Zhang. 2012. Technical Evaluation, Testing, and Validatiaon of the Usability of Electronic Health Records. US Department of Commerce, National Institute of Standards and Technology.
Dimitrios Markonis, Frederic Baroz, Rafael Luis Ruiz De Castaneda, Celia Boyer, and Henning Müller. 2013. User Tests for Assessing a Medical Image Retrieval System: A Pilot Study. In MEDINFO 2013. Studies in Health Technology and Informatics, Vol. 192. IOS Press, Amsterdam, Netherlands, 224–8.
Dimitrios Markonis, Markus Holzer, Frederic Baroz, Rafael Luis Ruiz De Castaneda, Célia Boyer, Georg Langs, and Henning Müller. 2015. User-oriented evaluation of a medical image retrieval system for radiologists. International Journal of Medical Informatics 84, 10 (2015), 774–783.
Luan Martins, Adriana Bueno, Alexandre Defelicibus, Rodrigo Drummond, Renan Valieris, Yu-Tao Zhu, Israel Da Silva, and Liang Zhao. 2023. WSI2ML – An Open-Source Whole Slide Image Annotation Software for Machine Learning Applications. In Anais do XXIX Simpósio Brasileiro de Sistemas Multimídia e Web (Ribeirão Preto/SP). SBC, Porto Alegre, RS, Brasil, 104–109. [link]
Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica 22, 3 (2012), 276–282.
J. Nielsen. 2020. 10 Usability Heuristics for User Interface Design. Nielsen Norman Group (2020). [link]
Jakob Nielsen and Rolf Molich. 1990. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. 249–256.
Kenneth Olbrish, Paul Shanken, Donna Rabe, Lorraine Steven, and Nicholas Irizarry. 2011. Four-year enterprise PACS support trend analysis. Journal of Digital Imaging 24 (2011), 284—-294.
Helen Petrie and Christopher Power. 2012. What do users really care about? A comparison of usability problems found by users and experts on highly interactive websites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2107–2116.
Lizawati Salahuddin, Zuraini Ismail, Ummi Rabaah Hashim, Nor Haslinda Ismail, Raja Rina Raja Ikram, Fiza Abdul Rahim, and Noor Hafizah Hassan. 2020. Health-care practitioner behaviours that influence unsafe use of hospital information systems. Health informatics journal 26, 1 (2020), 420–434.
Sandro Luis F. F. C. Silva, Marcelo Fornazin, and Rodrigo Pereira dos Santos. 2020. Analysis and Modeling of Emergent Systems in the Health Information System Domain. In XVI Brazilian Symposium on Information Systems. 1–8.
Marcílio Ferreira Souza-Júnior, Lívio Antonio Monteiro Amorim, Lucas Emanoel Paulino Silva Santos, Jorge Silva Correia-Neto, and Hugo Araujo Souza. 2019. Aprimoramento de Interfaces de Usuários de Sistemas de Informação em Saúde no Atendimento Pré-hospitalar na Ótica da Consciência Situacional. iSys-Brazilian Journal of Information Systems 12, 4 (2019), 98–116.
US National Electrical Manufacturers Association. 2021. Digital Imaging and Communications in Medicine (DICOM) stanrdard.
Cleverson Vieira, Marcus Oliveira, Marcelo Guimarães, Leonardo Rocha, and Diego Dias. 2023. Applied Explainable Artificial Intelligence (XAI) in the classification of retinal images for support in the diagnosis of Glaucoma. In Anais do XXIX Simpósio Brasileiro de Sistemas Multimídia e Web (Ribeirão Preto/SP). SBC, Porto Alegre, RS, Brasil, 82–90. [link]. Acessado: 19-05-2021.
Misagh Zahiri Esfahani, Jamileh Farokhzadian, Kambiz Bahaadinbeigy, and Reza Khajouei. 2019. Factors influencing the selection of a picture archiving and communication system: A qualitative study. The International journal of health planning and management 34, 2 (2019), 780–793. [link]
M. Alhajeri and S.G.S. Shah. 2019. Limitations in and Solutions for Improving the Functionality of Picture Archiving and Communication System: an Exploratory Study of PACS Professionals’ Perspectives. Journal of Digital Imaging 32 (2019), 54–67.
Priya Darshini B, Deepan Chakkaravarthy N, Gokul B, and Sabari Maharaja B. 2023. A Web-based Dicom Image and Plane Viewer. In 2023 Fifth International Conference on Electrical, Computer and Communication Technologies (ICECCT). IEEE, 1–7.
Rade R Babić, Zoran Milošević, and Gordana Stanković-Babić. 2012. Web technology in health information system. Acta Facultatis Medicae Naissensis 29, 2 (2012), 81–87.
Mirza Mansoor Baig, Hamid GholamHosseini, Aasia A Moqeem, Farhaan Mirza, and Maria Lindén. 2017. A systematic review of wearable patient monitoring systems–current challenges and opportunities for clinical adoption. Journal of medical systems 41, 7 (2017), 1–9.
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
Cronin, B. Kane, and G. Doherty. 2021. A Qualitative Analysis of the Needs and Experiences of Hospital-based Clinicians when Accessing Medical Imaging. Journal of digital imaging (2021), 1––12. DOI: 10.1007/s10278-021-00446-1
Fábio A. C. da Silva, André P. Freire, and Marluce R. Pereira. 2022. Understanding Interaction and Organizational Issues in Radiology Information Systems: a Qualitative Study with Health Professionals. In XVIII Brazilian Symposium on Information Systems. –.
Camila Rodrigues Dias, Marluce Rodrigues Pereira, and Andre Pimenta Freire. 2017. Qualitative review of usability problems in health information systems for radiology. Journal of biomedical informatics 76 (2017), 19–33.
K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. MIT Press, Cambridge, MA, USA.
Misagh Zahiri Esfahani, Reza Khajouei, and Mohammad Reza Baneshi. 2018. Augmentation of the think aloud method with users’ perspectives for the selection of a picture archiving and communication system. Journal of Biomedical Informatics 80 (2018), 43–51.
Ina Geldermann, Christoph Grouls, Christiane Kuhl, Thomas M Deserno, and Cord Spreckelsen. 2013. Black box integration of computer-aided diagnosis into PACS deserves a second chance: results of a usability study concerning bone age assessment. Journal of Digital Imaging 6 (2013), 698—-708.
Hai K Huang. 2003. Enterprise PACS and image distribution. Computerized Medical Imaging and Graphics 27, 2-3 (2003), 241–253.
Fateme Rangraz Jeddi, Ehsan Nabovati, Reyhane Bigham, and Razieh Farrahi. 2020. Usability evaluation of a comprehensive national health information system: A heuristic evaluation. Informatics in Medicine Unlocked 19 (2020), 100332.
Wiard Jorritsma, Fokie Cnossen, Rudi A Dierckx, Matthijs Oudkerk, and Peter MA van Ooijen. 2016. Pattern mining of user interaction logs for a post-deployment usability evaluation of a radiology PACS client. International Journal of Medical Informatics 85, 1 (2016), 36–42.
Wiard Jorritsma, Fokie Cnossen, Rudi A Dierckx, Matthijs Oudkerk, and Peter MA Van Ooijen. 2016. Post-deployment usability evaluation of a radiology workstation. International Journal of Medical Informatics 85, 1 (2016), 28–35.
Wiard Jorritsma, Fokie Cnossen, and Peter MA van Ooijen. 2014. Merits of usability testing for PACS selection. International Journal of Medical Informatics 83, 1 (2014), 27–36.
Wiard Jorritsma, Fokie Cnossen, and Peter MA van Ooijen. 2015. Adaptive support for user interface customization: a study in radiology. International Journal of Human-Computer Studies 77 (2015), 1–9.
Brandan Kennedy, Ellen Kerns, Y Raymond Chan, Barbara S Chaparro, and Sarah D Fouquet. 2019. Safeuristics! Do Heuristic Evaluation Violation Severity Ratings Correlate with Patient Safety Severity Ratings for a Native Electronic Health Record Mobile Application? Applied clinical informatics 10, 02 (2019), 210–218.
Dasueran Kim, Peter Kang, Jungmin Yun, Sung-Hye Park, Jeong-Wook Seo, and Peom Park. 2014. Study on User Interface of Pathology Picture Archiving and Communication System. Healthcare Informatics Research 20, 1 (2014), 45–51.
Andre W Kushniruk and Vimla L Patel. 2004. Cognitive and usability engineering methods for the evaluation of clinical information systems. Journal of biomedical informatics 37, 1 (2004), 56–76.
J. Lazar, J. H. Feng, and H. Hochheiser. 2017. Research Methods in Human-Computer Interaction. Morgan Kaufmann is an imprint of Elsevier, Cambridge, MA, USA.
James R Lewis. 1995. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7, 1 (1995), 57–78.
Svetlana Z Lowry, Patricia Abbott, Michael C Gibbons, Svetlana Z Lowry, Robert North, Emily S Patterson, Matthew T Quinn, Mala Ramaiah, Robert M Schumacher, and Jiajie Zhang. 2012. Technical Evaluation, Testing, and Validatiaon of the Usability of Electronic Health Records. US Department of Commerce, National Institute of Standards and Technology.
Dimitrios Markonis, Frederic Baroz, Rafael Luis Ruiz De Castaneda, Celia Boyer, and Henning Müller. 2013. User Tests for Assessing a Medical Image Retrieval System: A Pilot Study. In MEDINFO 2013. Studies in Health Technology and Informatics, Vol. 192. IOS Press, Amsterdam, Netherlands, 224–8.
Dimitrios Markonis, Markus Holzer, Frederic Baroz, Rafael Luis Ruiz De Castaneda, Célia Boyer, Georg Langs, and Henning Müller. 2015. User-oriented evaluation of a medical image retrieval system for radiologists. International Journal of Medical Informatics 84, 10 (2015), 774–783.
Luan Martins, Adriana Bueno, Alexandre Defelicibus, Rodrigo Drummond, Renan Valieris, Yu-Tao Zhu, Israel Da Silva, and Liang Zhao. 2023. WSI2ML – An Open-Source Whole Slide Image Annotation Software for Machine Learning Applications. In Anais do XXIX Simpósio Brasileiro de Sistemas Multimídia e Web (Ribeirão Preto/SP). SBC, Porto Alegre, RS, Brasil, 104–109. [link]
Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica 22, 3 (2012), 276–282.
J. Nielsen. 2020. 10 Usability Heuristics for User Interface Design. Nielsen Norman Group (2020). [link]
Jakob Nielsen and Rolf Molich. 1990. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. 249–256.
Kenneth Olbrish, Paul Shanken, Donna Rabe, Lorraine Steven, and Nicholas Irizarry. 2011. Four-year enterprise PACS support trend analysis. Journal of Digital Imaging 24 (2011), 284—-294.
Helen Petrie and Christopher Power. 2012. What do users really care about? A comparison of usability problems found by users and experts on highly interactive websites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2107–2116.
Lizawati Salahuddin, Zuraini Ismail, Ummi Rabaah Hashim, Nor Haslinda Ismail, Raja Rina Raja Ikram, Fiza Abdul Rahim, and Noor Hafizah Hassan. 2020. Health-care practitioner behaviours that influence unsafe use of hospital information systems. Health informatics journal 26, 1 (2020), 420–434.
Sandro Luis F. F. C. Silva, Marcelo Fornazin, and Rodrigo Pereira dos Santos. 2020. Analysis and Modeling of Emergent Systems in the Health Information System Domain. In XVI Brazilian Symposium on Information Systems. 1–8.
Marcílio Ferreira Souza-Júnior, Lívio Antonio Monteiro Amorim, Lucas Emanoel Paulino Silva Santos, Jorge Silva Correia-Neto, and Hugo Araujo Souza. 2019. Aprimoramento de Interfaces de Usuários de Sistemas de Informação em Saúde no Atendimento Pré-hospitalar na Ótica da Consciência Situacional. iSys-Brazilian Journal of Information Systems 12, 4 (2019), 98–116.
US National Electrical Manufacturers Association. 2021. Digital Imaging and Communications in Medicine (DICOM) stanrdard.
Cleverson Vieira, Marcus Oliveira, Marcelo Guimarães, Leonardo Rocha, and Diego Dias. 2023. Applied Explainable Artificial Intelligence (XAI) in the classification of retinal images for support in the diagnosis of Glaucoma. In Anais do XXIX Simpósio Brasileiro de Sistemas Multimídia e Web (Ribeirão Preto/SP). SBC, Porto Alegre, RS, Brasil, 82–90. [link]. Acessado: 19-05-2021.
Misagh Zahiri Esfahani, Jamileh Farokhzadian, Kambiz Bahaadinbeigy, and Reza Khajouei. 2019. Factors influencing the selection of a picture archiving and communication system: A qualitative study. The International journal of health planning and management 34, 2 (2019), 780–793. [link]
Publicado
14/10/2024
Como Citar
SILVA, Fábio Ap. Cândido da; FREIRE, André Pimenta; PEREIRA, Marluce Rodrigues.
Acceptance and Usability of Complex Medical Systems: A Study with Radiology Professionals. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 30. , 2024, Juiz de Fora/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 28-36.
DOI: https://doi.org/10.5753/webmedia.2024.242026.
