Index of User Experience in immersive Virtual Reality: a proposal for a standardized questionnaire
Abstract
Introduction: Considering particular aspects of the User Experience (UX) in immersive Virtual Reality (VR), psychological constructs such as presence and cybersickness transcend abstract conceptualizations: they have a practical impact on the user-technology interaction. Objective: This research presents the development and empirical evaluation of the index of User Experience in immersive Virtual Reality (iUXVR) for Portuguese-speaking contexts. Methodology or Steps: The creation process involved proposing items based on a content analysis and selecting the best items through expert evaluations. After a pilot study, an experiment with 77 participants resulted in the collection of 154 questionnaires, which were analyzed using Partial Least Squares Structural Equation Modeling. Results: Comprising 25 items distributed across fve key components – usability, aesthetics, sense of presence, VR sickness, and emotions – the final version of the iUXVR presents sufficient evidence of validity and a stable structural model, supporting its applicability.
Keywords:
User Experience, Virtual Reality, Evaluation, Standardized Questionnaire
References
AERA; APA; NCME (2014). Standards for Educational and Psychological Testing. American Educational Research Association, Washington, DC, EUA.
Barbosa, S. D. J., Silva, B. S. d., Silveira, M. S., Gasparini, I., Darin, T., e Barbosa, G. D. J. (2021). Interação Humano-Computador e Experiência do Usuário. autopublicação.
Becker, J.-M., Klein, K., e Wetzels, M. (2012). Hierarchical latent variable models in pls-sem: Guidelines for using reflective-formative type models. Long Range Planning, 45(5):359–394.
Carvalho, M. R. d., Costa, R. T. d., e Nardi, A. E. (2011). Simulator Sickness Questionnaire: tradução e adaptação transcultural. Jornal Brasileiro de Psiquiatria, 60(4):0–5.
Chang, E., Kim, H. T., e Yoo, B. (2020). Virtual Reality Sickness: A Review of Causes and Measurements. International Journal of Human-Computer Interaction, 36(17):1658–1682.
Cheng, L. K., Chieng, M. H., e Chieng, W. H. (2014). Measuring virtual experience in a three-dimensional virtual reality interactive simulator environment: A structural equation modeling approach. Virtual Reality, 18(3):173–188.
Cota, M. P., Thomaschewski, J., Schrepp, M., e Gonçalves, R. (2014). Efficient Measurement of the User Experience . A Portuguese Version. Procedia - Procedia Computer Science, 27:491–498.
Darin, T., Coelho, B., e Borges, B. (2019). Which Instrument Should I Use? Supporting Decision-Making About the Evaluation of User Experience. In Design, User Experience, and Usability. Practice and Case Studies, pages 49–67, Cham, Switzerland. Springer International Publishing.
Davis, S., Nesbitt, K., e Nalivaiko, E. (2014). A Systematic Review of Cybersickness. In Proceedings of the Conference on Interactive Entertainment, pages 1–9, New York, NY, USA. Association for Computing Machinery.
Grassini, S. e Laumann, K. (2020). Questionnaire measures and physiological correlates of presence: A systematic review. Frontiers in Psychology, 11:349.
Hair, J. e Alamer, A. (2022). Partial least squares structural equation modeling (pls-sem) in second language and education research: Guidelines using an applied example. Research Methods in Applied Linguistics, 1(3).
Hair, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., e Ray, S. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: A workbook. Springer, Cham, Switzerland.
Hassenzahl, M. (2003). The thing and i: Understanding the relationship between user and product. In Blythe, M. A., Overbeeke, K., Monk, A. F., e Wright, P. C., editors, Funology: From Usability to Enjoyment, pages 31–42. Springer Netherlands, Dordrecht, Netherlands.
Hassenzahl, M., Burmester, M., e Koller, F. (2003). AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In Szwillus, G. e Ziegler, J., editors, Mensch & Computer 2003: Interaktion in Bewegung, pages 187–196. Vieweg Teubner Verlag, Wiesbaden.
ISO (2018). ISO 9241-11:2018 - Ergonomics of human-system interaction - part 11: Usability: Defnitions and concepts. Standard, ISO, Geneva, Switzerland.
ISO (2019). ISO 9241-210:2019 - Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. Standard, ISO, Geneva, Switzerland.
Izard, C. E. (1977). Theories of Emotion and Emotion-Behavior Relationships, pages 19–42. Springer US, Boston, MA.
Izard, C. E. (2007). Basic Emotions, Natural Kinds, Emotion Schemas, and a New Paradigm. Perspectives on Psychological Science, 2(3):260–280.
Kim, Y. M., Rhiu, I., e Yun, M. H. (2020). A Systematic Review of a Virtual Reality System from the Perspective of User Experience. International Journal of Human-Computer Interaction, 36(10):893–910.
Kitchenham, B. A., Budgen, D., e Brereton, O. P. (2011). Using mapping studies as the basis for further research – A participant-observer case study. Information and Software Technology, 53(6):638–651.
Lavie, T. e Tractinsky, N. (2004). Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human Computer Studies, 60(3):269–298.
Law, E. L. C., Van Schaik, P., e Roto, V. (2014). Attitudes towards user experience (UX) measurement. International Journal of Human Computer Studies, 72(6):526–541.
Liengaard, B. D. (2024). Measurement invariance testing in partial least squares structural equation modeling. Journal of Business Research, 177:114581.
Lourenço, D. F., Carmona, E. V., e de Moraes Lopes, M. H. B. (2022). Translation and cross-cultural adaptation of the System Usability Scale to Brazilian Portuguese. Aquichan, 22(2).
Mahlke, S. (2008). User Experience of Interaction with Technical Systems. Doktor der ingenieurwissenschaften, Technische Universität Berlin, Fakultät V - Verkehrsund Maschinensysteme, Alemanha.
Maia, C. L. B. e Furtado, E. S. (2016). A systematic review about user experience evaluation. In Marcus, A., editor, Design, User Experience, and Usability: Design Thinking and Methods, pages 445–455, Cham, Switzerland. Springer International Publishing.
Marques, L., Barcellos, M. P., Gadelha, B., e Conte, T. (2024). Characterizing ux assessment in the context of immersive experiences: A systematic mapping study. International Journal of Human–Computer Interaction, 0(0):1–17.
Martins, A. I., Rosa, A. F., Queirós, A., Silva, A., e Rocha, N. P. (2015). European Portuguese validation of the System Usability Scale (SUS). Procedia - Procedia Computer Science, 67:293–300.
Martirosov, S. e Kopecek, P. (2017). Cyber sickness in virtual reality - Literature review. In Katalinic, B., editor, Proceedings of the DAAAM International Symposium, pages 718–726, Vienna, Austria. DAAAM International.
Melo, B. e Darin, T. (2019). Scope and definition of user experience in brazil: a survey to explore community’s perspectives. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems, pages 1–11, New York, NY, USA. ACM.
Minge, M., Thüring, M., Wagner, I., e Kuhr, C. V. (2016). The meCUE questionnaire: A modular tool for measuring user experience. In Soares, M., Falcão, C., e Ahram, T. Z., editors, Advances in Ergonomics Modeling, Usability and Special Populations, pages 115–128, Cham, Switzerland. Springer International Publishing.
Moraes, R. (1999). Análise de conteúdo. Revista Educação, 22(27).
Nielsen, J. (1993). Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
Norman, D. A. (2004). Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, New York, NY, USA.
Nur, A. I., Santoso, H. B., e Putra, P. O. H. (2021). The method and metric of user experience evaluation: A systematic literature review. In Proceedings of the International Conference on Software and Computer Applications, pages 307–317, New York, NY, EUA. ACM.
Preece, J., Rogers, Y., e Sharp, H. (2015). Interaction Design: Beyond Human-Computer Interaction. Wiley, Hoboken, NJ, 4 edition.
Reise, S. P., Bonifay, W. E., e Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2):129–140.
Research e Markets (2025). Virtual Reality Market Report by Component, Type, Application, Technology, Regions and Company Analysis 2025-2033. Report, Research and Markets, s.l.
Reynolds, C. R., Altmann, R. A., e Allen, D. N. (2021). Mastering Modern Psychological Testing: theory and methods. Springer, Cham, Switzerland, 2 edition.
Rivero, L. e Conte, T. (2017). A Systematic Mapping Study on Research Contributions on UX Evaluation Technologies. In Proceedings of the Brazilian Symposium on Human Factors in Computing Systems, pages 1–10, New York, NY, USA. ACM.
Russo, D. e Stol, K. J. (2021). PLS-SEM for software engineering research: An introduction and survey. ACM Computing Surveys, 54(4).
Sauro, J. e Lewis, J. (2016). Quantifying the user experience: practical statistics for user research. Morgan Kaufmann, Massachusetts, USA.
Schrepp, M., Hinderks, A., e Thomaschewski, J. (2014). Applying the User Experience Questionnaire (UEQ) in different evaluation scenarios. In Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience, pages 383–392, Cham. Springer International Publishing.
Schrepp, M. e Thomaschewski, J. (2019). Design and validation of a framework for the creation of user experience questionnaires. International Journal of Interactive Multimedia and Artificial Intelligence, 5(7):88–95.
Sevinc, V. e Ilker, B. M. (2020). Psychometric evaluation of simulator sickness questionnaire and its variants as a measure of cybersickness in consumer virtual environments. Applied Ergonomics, 82:1–12.
Shin, D. H., Biocca, F., e Choo, H. (2013). Exploring the user experience of three-dimensional virtual learning environments. Behaviour and Information Technology, 32(2):203–214.
Silva, G. R., Donat, J. C., Rigoli, M. M., de Oliveira, F. R., e Kristensen, C. H. (2016). A questionnaire for measuring presence in virtual environments: factor analysis of the presence questionnaire and adaptation into Brazilian Portuguese. Virtual Reality, 20(4):237–242.
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B - Biological Sciences, 364(1535):3549–3557.
Tcha-Tokey, K., Christmann, O., Loup-Escande, E., e Richir, S. (2016). Proposition and Validation of a Questionnaire to Measure the User Experience in Immersive Virtual Environments. International Journal of Virtual Reality, 16(1):33–48.
Tcha-Tokey, K., Loup-Escande, E., Christmann, O., Canac, G., Farin, F., e Richir, S. (2018). Towards a user experience model in immersive virtual environments. Advances in Human-Computer Interaction, 2018:1–10.
Vasconcelos-Raposo, J., Melo, M., Barbosa, L., Teixeira, C., Cabral, L., e Bessa, M. (2021). Assessing presence in virtual environments: adaptation of the psychometric properties of the Presence Questionnaire to the Portuguese populations. Behaviour and Information Technology, 40(13):1417–1427.
Vermeeren, A., Law, E., Roto, V., Obrist, M., Hoonhout, J., e Väänänen-Vainio-Mattila, K. (2010). User experience evaluation methods: Current state and development needs. In Proceedings of the Nordic Conference on Human-Computer Interaction, pages 521–530, New York, NY, USA. ACM.
Witmer, B. G., Jerome, C. J., e Singer, M. J. (2005). The Factor Structure of the Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 14(3):298–312.
Witmer, B. G. e Singer, M. J. (1998). Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 7(3):225–240.
Barbosa, S. D. J., Silva, B. S. d., Silveira, M. S., Gasparini, I., Darin, T., e Barbosa, G. D. J. (2021). Interação Humano-Computador e Experiência do Usuário. autopublicação.
Becker, J.-M., Klein, K., e Wetzels, M. (2012). Hierarchical latent variable models in pls-sem: Guidelines for using reflective-formative type models. Long Range Planning, 45(5):359–394.
Carvalho, M. R. d., Costa, R. T. d., e Nardi, A. E. (2011). Simulator Sickness Questionnaire: tradução e adaptação transcultural. Jornal Brasileiro de Psiquiatria, 60(4):0–5.
Chang, E., Kim, H. T., e Yoo, B. (2020). Virtual Reality Sickness: A Review of Causes and Measurements. International Journal of Human-Computer Interaction, 36(17):1658–1682.
Cheng, L. K., Chieng, M. H., e Chieng, W. H. (2014). Measuring virtual experience in a three-dimensional virtual reality interactive simulator environment: A structural equation modeling approach. Virtual Reality, 18(3):173–188.
Cota, M. P., Thomaschewski, J., Schrepp, M., e Gonçalves, R. (2014). Efficient Measurement of the User Experience . A Portuguese Version. Procedia - Procedia Computer Science, 27:491–498.
Darin, T., Coelho, B., e Borges, B. (2019). Which Instrument Should I Use? Supporting Decision-Making About the Evaluation of User Experience. In Design, User Experience, and Usability. Practice and Case Studies, pages 49–67, Cham, Switzerland. Springer International Publishing.
Davis, S., Nesbitt, K., e Nalivaiko, E. (2014). A Systematic Review of Cybersickness. In Proceedings of the Conference on Interactive Entertainment, pages 1–9, New York, NY, USA. Association for Computing Machinery.
Grassini, S. e Laumann, K. (2020). Questionnaire measures and physiological correlates of presence: A systematic review. Frontiers in Psychology, 11:349.
Hair, J. e Alamer, A. (2022). Partial least squares structural equation modeling (pls-sem) in second language and education research: Guidelines using an applied example. Research Methods in Applied Linguistics, 1(3).
Hair, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., e Ray, S. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: A workbook. Springer, Cham, Switzerland.
Hassenzahl, M. (2003). The thing and i: Understanding the relationship between user and product. In Blythe, M. A., Overbeeke, K., Monk, A. F., e Wright, P. C., editors, Funology: From Usability to Enjoyment, pages 31–42. Springer Netherlands, Dordrecht, Netherlands.
Hassenzahl, M., Burmester, M., e Koller, F. (2003). AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In Szwillus, G. e Ziegler, J., editors, Mensch & Computer 2003: Interaktion in Bewegung, pages 187–196. Vieweg Teubner Verlag, Wiesbaden.
ISO (2018). ISO 9241-11:2018 - Ergonomics of human-system interaction - part 11: Usability: Defnitions and concepts. Standard, ISO, Geneva, Switzerland.
ISO (2019). ISO 9241-210:2019 - Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. Standard, ISO, Geneva, Switzerland.
Izard, C. E. (1977). Theories of Emotion and Emotion-Behavior Relationships, pages 19–42. Springer US, Boston, MA.
Izard, C. E. (2007). Basic Emotions, Natural Kinds, Emotion Schemas, and a New Paradigm. Perspectives on Psychological Science, 2(3):260–280.
Kim, Y. M., Rhiu, I., e Yun, M. H. (2020). A Systematic Review of a Virtual Reality System from the Perspective of User Experience. International Journal of Human-Computer Interaction, 36(10):893–910.
Kitchenham, B. A., Budgen, D., e Brereton, O. P. (2011). Using mapping studies as the basis for further research – A participant-observer case study. Information and Software Technology, 53(6):638–651.
Lavie, T. e Tractinsky, N. (2004). Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human Computer Studies, 60(3):269–298.
Law, E. L. C., Van Schaik, P., e Roto, V. (2014). Attitudes towards user experience (UX) measurement. International Journal of Human Computer Studies, 72(6):526–541.
Liengaard, B. D. (2024). Measurement invariance testing in partial least squares structural equation modeling. Journal of Business Research, 177:114581.
Lourenço, D. F., Carmona, E. V., e de Moraes Lopes, M. H. B. (2022). Translation and cross-cultural adaptation of the System Usability Scale to Brazilian Portuguese. Aquichan, 22(2).
Mahlke, S. (2008). User Experience of Interaction with Technical Systems. Doktor der ingenieurwissenschaften, Technische Universität Berlin, Fakultät V - Verkehrsund Maschinensysteme, Alemanha.
Maia, C. L. B. e Furtado, E. S. (2016). A systematic review about user experience evaluation. In Marcus, A., editor, Design, User Experience, and Usability: Design Thinking and Methods, pages 445–455, Cham, Switzerland. Springer International Publishing.
Marques, L., Barcellos, M. P., Gadelha, B., e Conte, T. (2024). Characterizing ux assessment in the context of immersive experiences: A systematic mapping study. International Journal of Human–Computer Interaction, 0(0):1–17.
Martins, A. I., Rosa, A. F., Queirós, A., Silva, A., e Rocha, N. P. (2015). European Portuguese validation of the System Usability Scale (SUS). Procedia - Procedia Computer Science, 67:293–300.
Martirosov, S. e Kopecek, P. (2017). Cyber sickness in virtual reality - Literature review. In Katalinic, B., editor, Proceedings of the DAAAM International Symposium, pages 718–726, Vienna, Austria. DAAAM International.
Melo, B. e Darin, T. (2019). Scope and definition of user experience in brazil: a survey to explore community’s perspectives. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems, pages 1–11, New York, NY, USA. ACM.
Minge, M., Thüring, M., Wagner, I., e Kuhr, C. V. (2016). The meCUE questionnaire: A modular tool for measuring user experience. In Soares, M., Falcão, C., e Ahram, T. Z., editors, Advances in Ergonomics Modeling, Usability and Special Populations, pages 115–128, Cham, Switzerland. Springer International Publishing.
Moraes, R. (1999). Análise de conteúdo. Revista Educação, 22(27).
Nielsen, J. (1993). Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
Norman, D. A. (2004). Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, New York, NY, USA.
Nur, A. I., Santoso, H. B., e Putra, P. O. H. (2021). The method and metric of user experience evaluation: A systematic literature review. In Proceedings of the International Conference on Software and Computer Applications, pages 307–317, New York, NY, EUA. ACM.
Preece, J., Rogers, Y., e Sharp, H. (2015). Interaction Design: Beyond Human-Computer Interaction. Wiley, Hoboken, NJ, 4 edition.
Reise, S. P., Bonifay, W. E., e Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2):129–140.
Research e Markets (2025). Virtual Reality Market Report by Component, Type, Application, Technology, Regions and Company Analysis 2025-2033. Report, Research and Markets, s.l.
Reynolds, C. R., Altmann, R. A., e Allen, D. N. (2021). Mastering Modern Psychological Testing: theory and methods. Springer, Cham, Switzerland, 2 edition.
Rivero, L. e Conte, T. (2017). A Systematic Mapping Study on Research Contributions on UX Evaluation Technologies. In Proceedings of the Brazilian Symposium on Human Factors in Computing Systems, pages 1–10, New York, NY, USA. ACM.
Russo, D. e Stol, K. J. (2021). PLS-SEM for software engineering research: An introduction and survey. ACM Computing Surveys, 54(4).
Sauro, J. e Lewis, J. (2016). Quantifying the user experience: practical statistics for user research. Morgan Kaufmann, Massachusetts, USA.
Schrepp, M., Hinderks, A., e Thomaschewski, J. (2014). Applying the User Experience Questionnaire (UEQ) in different evaluation scenarios. In Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience, pages 383–392, Cham. Springer International Publishing.
Schrepp, M. e Thomaschewski, J. (2019). Design and validation of a framework for the creation of user experience questionnaires. International Journal of Interactive Multimedia and Artificial Intelligence, 5(7):88–95.
Sevinc, V. e Ilker, B. M. (2020). Psychometric evaluation of simulator sickness questionnaire and its variants as a measure of cybersickness in consumer virtual environments. Applied Ergonomics, 82:1–12.
Shin, D. H., Biocca, F., e Choo, H. (2013). Exploring the user experience of three-dimensional virtual learning environments. Behaviour and Information Technology, 32(2):203–214.
Silva, G. R., Donat, J. C., Rigoli, M. M., de Oliveira, F. R., e Kristensen, C. H. (2016). A questionnaire for measuring presence in virtual environments: factor analysis of the presence questionnaire and adaptation into Brazilian Portuguese. Virtual Reality, 20(4):237–242.
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B - Biological Sciences, 364(1535):3549–3557.
Tcha-Tokey, K., Christmann, O., Loup-Escande, E., e Richir, S. (2016). Proposition and Validation of a Questionnaire to Measure the User Experience in Immersive Virtual Environments. International Journal of Virtual Reality, 16(1):33–48.
Tcha-Tokey, K., Loup-Escande, E., Christmann, O., Canac, G., Farin, F., e Richir, S. (2018). Towards a user experience model in immersive virtual environments. Advances in Human-Computer Interaction, 2018:1–10.
Vasconcelos-Raposo, J., Melo, M., Barbosa, L., Teixeira, C., Cabral, L., e Bessa, M. (2021). Assessing presence in virtual environments: adaptation of the psychometric properties of the Presence Questionnaire to the Portuguese populations. Behaviour and Information Technology, 40(13):1417–1427.
Vermeeren, A., Law, E., Roto, V., Obrist, M., Hoonhout, J., e Väänänen-Vainio-Mattila, K. (2010). User experience evaluation methods: Current state and development needs. In Proceedings of the Nordic Conference on Human-Computer Interaction, pages 521–530, New York, NY, USA. ACM.
Witmer, B. G., Jerome, C. J., e Singer, M. J. (2005). The Factor Structure of the Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 14(3):298–312.
Witmer, B. G. e Singer, M. J. (1998). Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 7(3):225–240.
Published
2025-09-08
How to Cite
CHEIRAN, Jean F. P.; BANDEIRA, Denise R.; PIMENTA, Marcelo S..
Index of User Experience in immersive Virtual Reality: a proposal for a standardized questionnaire. In: BRAZILIAN SYMPOSIUM ON HUMAN FACTORS IN COMPUTATIONAL SYSTEMS (IHC), 24. , 2025, Belo Horizonte/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 1049-1070.
DOI: https://doi.org/10.5753/ihc.2025.10750.
