A Usability and User Experience Evaluation Technology for Touchable Holographic Solutions

Abstract


Introduction: Touchable Holographic Solutions (THS) present novel challenges for evaluating usability and user experience (UX), particularly in relation to immersion and presence. Traditional instruments may not fully capture the specific aspects involved in interacting with holographic content. Objective: This work aims to present UUXE-ToH, a questionnaire specifically designed to evaluate usability and UX in THS contexts. Methodology or Steps: The questionnaire was developed through a multi-step process that included systematic mapping studies, exploratory search, expert validation, and studies with end users. The final version of the instrument comprises 56 items grouped into 19 UX aspects. Results: Empirical studies indicate that UUXE-ToH is a valid and reliable instrument. It identifies more usability and UX issues compared to existing instruments and is also rated more positively in terms of ease of use and acceptance.

Keywords: Touchable Holography, Usability, User eXperience (UX), Evaluation

References

Alexandrovsky, D., Putze, S., Schwind, V., Mekler, E. D., Smeddinck, J. D., Kahl, D., Krüger, A., e Malaka, R. (2021). Evaluating User Experiences in Mixed Reality. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–5, Yokohama Japan. ACM.

Basili, V. R., Caldiera, G., e Rombach, H. D. (1994). The Goal Question Metric Approach.

Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., e Young, S. L. (2018). Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Frontiers in Public Health, 6:149.

Bouwel, T. V. (2025). Cubism Presskit. Retrieved from [link] on July 28, 2025.

Brooke, J. (1996). SUS: A ’Quick and Dirty’ Usability Scale. In Usability Evaluation In Industry, pages 207–212. CRC Press, London, 1st edition.

Campos, T., Castello, M., Valentim, N., e Damasceno, E. (2024a). An Updated Systematic Mapping Study on Usability and User Experience Evaluation of Touchable Holograms: Technical Report. Technical report, Federal University of Paraná, Curitiba, Brazil.

Campos, T., Damasceno, E., e Valentim, N. (2024b). Usability and User Experience Questionnaire Evaluation and Evolution for Touchable Holography. In Proceedings of the 26th International Conference on Enterprise Information Systems, pages 449–460, Angers, France. SCITEPRESS - Science and Technology Publications.

Campos, T., Delabrida, S., Damasceno, E., e Valentim, N. (2025a). Evaluating Performance and Acceptance of the UUXE-ToH Questionnaire for Touchable Holographic Solutions. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS, pages 641–648. SciTePress.

Campos, T., Delabrida, S., e Valentim, N. (2024c). Avaliação da Usabilidade e da Experiência do Usuário em Realidade Aumentada e Virtual. In Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais (IHC), pages 7–8. SBC.

Campos, T., Delabrida, S., e Valentim, N. (2025b). Avaliação da Usabilidade e da Experiência do Usuário em Realidade Virtual e Aumentada. In Minicursos IHC 2024 - Fundamentos e Práticas para Experiências Digitais Acessíveis, Inclusivas e Eticamente Responsáveis, Perspectivas e Práticas Contemporâneas em IHC, pages 39–62. Sociedade Brasileira de Computação (SBC), Porto Alegre, Brasil.

Campos, T., Delabrida, S., e Valentim, N. (2025c). Usability and UX Evaluation in a Mixed Reality Puzzle Game Using Questionnaires. Journal on Interactive Systems, 16(1).

Campos, T., Valentim, N., e Damasceno, E. (2023). A Systematic Mapping Study on Usability and User Experience Evaluation of Touchable Holograms: Technical Report. Technical report, Federal University of Paraná, Curitiba, Brazil.

Campos, T. P. D., Damasceno, E. F., e Valentim, N. M. C. (2022a). Porifera: A Collaborative Tool to Support Systematic Literature Review and Systematic Mapping Study. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering, SBES ’22, pages 452–457, New York, NY, USA. Association for Computing Machinery.

Campos, T. P. d., Damasceno, E. F., e Valentim, N. M. C. (2022b). Proposal and Evaluation of a Collaborative IS to Support Systematic Reviews and Mapping Studies. In Proceedings of the XVIII Brazilian Symposium on Information Systems, SBSI ’22, pages 1–8, New York, NY, USA. Association for Computing Machinery.

Corbin, J. e Strauss, A. (2014). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications, London, UK.

Costa, F. J. D. (2021). Mensuracao E Desenvolvimento De Escalas. Ciência Moderna, Rio de Janeiro, 1st edition.

De Souza, A. C. R., Mariano, P. A. D. L., Guerino, G. C., Chaves, A. P., e Valentim, N. M. C. (2023). Technologies for Hedonic Aspects Evaluation in Text-based Chatbots: A Systematic Mapping Study. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems, pages 1–11, Maceió Brazil. ACM.

DeVellis, R. F. e Thorpe, C. T. (2022). Scale development: theory and applications. SAGE Publications, Inc, Thousand Oaks, California, 5th edition.

Filho, G. K., Guerino, G. C., e Valentim, N. M. C. (2023). Usability and User Experience of Multi-touch Systems: A Systematic Mapping Study and Benchmark. Journal on Interactive Systems, 14(1):292–316.

Gao, M., Kortum, P., e Oswald, F. (2018). Psychometric Evaluation of the USE (Usefulness, Satisfaction, and Ease of use) Questionnaire for Reliability and Validity. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62(1):1414–1418.

Haddad, F. B. B., Filho, W. A. R., Ramos, V. M., Corrêa, C. G., e Peres, L. M. (2024). Mapeamento sistemático da literatura de jogos educacionais destinados ao ensino e aprendizagem de Engenharia de Software: uma análise do estado da arte. In Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames), pages 1256–1269. SBC.

Hart, S. G. e Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Hancock, P. A. e Meshkati, N., editors, Advances in Psychology, volume 52 of Human Mental Workload, pages 139–183. North-Holland, Amsterdan, Netherlands.

Hassenzahl, M. (2008). The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interaction, 19(4):319–349.

Hassenzahl, M., Burmester, M., e Koller, F. (2003). AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In Szwillus, G. e Ziegler, J., editors, Mensch & Computer 2003: Interaktion in Bewegung, pages 187–196. Vieweg+Teubner Verlag, Wiesbaden.

Hassenzahl, M., Platz, A., Burmester, M., e Lehner, K. (2000). Hedonic and ergonomic quality aspects determine a software’s appeal. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, CHI ’00, pages 201–208, New York, NY, USA. Association for Computing Machinery.

Hoshi, T., Takahashi, M., Nakatsuma, K., e Shinoda, H. (2009). Touchable holography. In ACM SIGGRAPH 2009 Emerging Technologies, SIGGRAPH ’09, pages 1–1, New York, New York, USA. ACM Press.

ISO (2011). ISO/IEC 25010:2011 - Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. Technical report, International Organization for Standardization, Geneva, Switzerland.

ISO (2018). ISO 9241-11:2018 Ergonomics of human-system interaction — Part 11: Usability: Defnitions and concepts. Technical report, International Organization for Standardization, Geneva, Switzerland.

Izo, F., Vereau, L. E. S. P., Pirovani, J. P. C., Oliveira, E., e Badue, C. (2023). An Intelligent Report Generator for Chemical Documents. In Proceedings of the XIX Brazilian Symposium on Information Systems, pages 276–283, Maceió Brazil. ACM.

Kitchenham, B. e Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. Technical report, Keele University and University of Durham. ISBN: 1595933751 ISSN: 00010782 eprint: 1304.1186.

Kitchenham, B. A., Budgen, D., e Brereton, P. (2016). Evidence-based software engineering and systematic reviews. Chapman & Hall/CRC innovations in software engineering and software development. CRC Press, Boca Raton. OCLC: ocn932588149.

Laugwitz, B., Held, T., e Schrepp, M. (2008). Construction and Evaluation of a User Experience Questionnaire. In Holzinger, A., editor, HCI and Usability for Education and Work, Lecture Notes in Computer Science, pages 63–76, Graz, Austria. Springer.

Lewis, J. R. e Sauro, J. (2009). The Factor Structure of the System Usability Scale. In Kurosu, M., editor, Human Centered Design, pages 94–103, Berlin, Heidelberg. Springer.

Lewis, J. R. e Sauro, J. (2017). Revisiting the Factor Structure of the System Usability Scale - JUX. JUX - The Journal of User Experience, 12(4):183–192.

Mafra, S. N., Barcelos, R. F., e Travassos, G. H. (2006). Aplicando uma Metodologia Baseada em Evidência na Defnição de Novas Tecnologias de Software. In Anais do Simpósio Brasileiro de Engenharia de Software (SBES), pages 239–254. SBC.

Marchionini, G. (2006). Exploratory search: from fnding to understanding. Communications of the ACM, 49(4):41–46.

Mariano, P. A. d. L., Souza, A. C. R. d., Guerino, G. C., Chaves, A. P., e Valentim, N. M. C. (2024). A Systematic Mapping Study about Technologies for Hedonic Aspects Evaluation in Text-based Chatbots. Journal on Interactive Systems, 15(1):875–896.

Marques, L., Matsubara, P., Nakamura, W., Wiese, I., Zaina, L., e Conte, T. (2019). UX-Tips: A UX evaluation technique to support the identification of software application problems. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering, SBES ’19, pages 224–233, New York, NY, USA. Association for Computing Machinery.

Morville, P. (2004). User Experience Design. Retrieved from [link] January 20, 2023.

Nielsen, J. (2012). Usability 101: Introduction to Usability. Retrieved from [link] on December 14, 2020.

Petersen, K., Feldt, R., Mujtaba, S., e Mattsson, M. (2008). Systematic mapping studies in software engineering. 12th International Conference on Evaluation and Assessment in Software Engineering, EASE 2008, I(June):1–10.

Petersen, K., Vakkalanka, S., e Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.

Prado De Campos, T., Damasceno, E. F., e Valentim, N. M. C. (2024). Evaluating Usability and UX in Touchable Holographic Solutions: A Validation Study of the UUXE-ToH Questionnaire. International Journal of Human–Computer Interaction, pages 1–21.

Prado De Campos, T., Filgueiras Damasceno, E., e Malveira Costa Valentim, N. (2023). Avaliando um SI Colaborativo para Apoiar Revisões e Mapeamentos Sistemáticos da Literatura. iSys - Brazilian Journal of Information Systems, 16(1).

Shull, F., Carver, J., e Travassos, G. H. (2001). An empirical methodology for introducing software processes. ACM SIGSOFT Software Engineering Notes, 26(5):288–296.

Silva, D., Conte, T., e Valentim, N. (2024). A Systematic Mapping Study about Learner Experience Design in Computational Systems. Informatics in Education - An International Journal, 23(2):439–478.

Sinlapanuntakul, W., Korentsides, J., Collard, A., Skilton, K., e Chaparro, B. (2022). Touching Holograms: A Preliminary Evaluation of Mixed Reality Gestures.

Skarbez, R., Brooks, F. P., e Whitton, M. C. (2021). Immersion and Coherence: Research Agenda and Early Results. IEEE Transactions on Visualization and Computer Graphics, 27(10):3839–3850.

Skarbez, R., Smith, M., Sadagic, A., e Whitton, M. C., editors (2022). Presence and Beyond: Evaluating User Experience in AR/MR/VR. Frontiers Research Topics. Frontiers Media SA.

Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535):3549–3557.

Vieira, K. M. e Bressan, A. A. (2022). Construção e Validação de Instrumentos de Pesquisa de Survey: da Psicologia à Administração. Revista Administração em Diálogo - RAD, 24(3):7–27.

Witmer, B. G. e Singer, M. J. (1998). Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 7(3):225-240.

Zarour, M. e Alharbi, M. (2017). User experience framework that combines aspects, dimensions, and measurement methods. Cogent Engineering, 4(1):1421006.
Published
2025-09-08
CAMPOS, Thiago Prado de; DAMASCENO, Eduardo Filgueiras; VALENTIM, Natasha M. C.. A Usability and User Experience Evaluation Technology for Touchable Holographic Solutions. In: BRAZILIAN SYMPOSIUM ON HUMAN FACTORS IN COMPUTATIONAL SYSTEMS (IHC), 24. , 2025, Belo Horizonte/MG. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2025 . p. 838-861. DOI: https://doi.org/10.5753/ihc.2025.11932.