The Awareness Assessment Model: measuring the awareness and collaboration support over the participant’s perspective
Resumo
[Context] Awareness has been a valuable concept in Collaborative Systems since its formation, being an essential part of groupware. The efficient awareness mechanism ensures a better understanding and, consequently, a better projection of future actions; in contrast, the lack of these mechanisms undermines comprehension and prevents participants from projecting their work accordingly. [Problem] This is a multi-factorial problem, and finding a goodm starting point in the literature can be challenging for novice groupware designers; they must reinvent awareness from their own experience of what it is, how it works, and how it is used. [Goal] This work consists of establishing an assessment model for collaborative interfaces by analyzing the awareness mechanisms provided from the participant’s viewpoint. Our awareness assessment model developed adopting the statistical technique Item Response Theory (IRT) and considers the participant’s skill in understanding the awareness and the difficulty involved. [Results] The proposed assessment model allow us to measure the awareness support provided considering the collaboration, workspace, and contextual awareness perspectives. The results obtained were translated into an awareness support scale and three levels of quality were defined.
Referências
Agogo, D. and Hess, T. J. (2018). “how does tech make you feel?” a review and examination of negative affective responses to technology use. European Journal of Information Systems, 27(5):570–599.
Antunes, P., Herskovic, V., Ochoa, S. F., and Pino, J. A. (2014). Reviewing the quality of awareness support in collaborative applications. Journal of Systems and Software, 89(1):146–169.
Bailey, K. D. (1994). Typologies and taxonomies: An introduction to classification techniques, volume 102. Sage.
Baker, F. B. (2001). The basics of item response theory. ERIC.
Baker, F. B. and Kim, S.-H. (2004). Item response theory: Parameter estimation techniques. CRC press.
Baker, F. B. and Kim, S.-H. (2017). The basics of item response theory using R. Springer.
Basili, V. (1992). Software modeling and measurement: The goal/question/metric paradigm. Technical Report.
Beecham, S., Hall, T., Britton, C., Cottee, M., and Rainer, A. (2005). Using an expert panel to validate a requirements process improvement model. Journal of Systems and Software, 76(3):251–275.
Collazos, C. A., Gutierrez, F. L., Gallardo, J., Ortega, M., Fardoun, H. M., and Molina, A. I. (2019). Descriptive theory of awareness for groupware development. Journal of Ambient Intelligence and Humanized Computing, 10(12):4789–4818.
de Andrade, D. F., Tavares, H. R., and da Cunha Valle, R. (2000). Teoria da resposta ao item: conceitos e aplicações. ABE, Sao Paulo.
DeVellis, R. F. (2016). Scale development: Theory and applications, volume 26 of Applied Social Research Methods 26. Sage Publications, Inc, 4th edition.
Dourish, P. and Bellotti, V. (1992). Awareness and coordination in shared workspaces. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work, CSCW ’92, pages 107–114, New York, NY, USA. Association for Computing Machinery
Gallardo, J., Bravo, C., and Molina, A. I. (2018). A framework for the descriptive specification of awareness support in multimodal user interfaces for collaborative activities. Journal on Multimodal User Interfaces, 12(2):145–159.
Gross, T. (2013). Supporting effortless coordination: 25 years of awareness research. Computer Supported Cooperative Work, 22(4-6):425–474.
Hinkelmann, K. and Kempthorne, O. (2005). Design and analysis of experiments: Advanced experimental design. Wiley-Interscience.
Hummel, D., Schacht, S., and Maedche, A. (2016). Determinants of multi-channel behavior: Exploring avenues for future research in the services industry. In ICIS.
Khalilijafarabad, A., Helfert, M., and Ge, M. (2016). Developing a data quality research taxonomy-an organizational perspective. In ICIQ, pages 176–186.
Mantau, M. J. and Benitti, F. B. V. (2022a). Awareness support in collaborative system: Reviewing last 10 years of cscw research. In 5th IEEE International Conference on Computer Supported Cooperative Work in Design, CSCWD 2022, Hangzhou, China, May 4-6, 2022, pages 564–569. IEEE.
Mantau, M. J. and Benitti, F. B. V. (2022b). Towards an awarenes taxonomy. In 25th IEEE International Conference on Computer Supported Cooperative Work in Design, CSCWD 2022, Hangzhou, China, May 4-6, 2022, pages 495–500. IEEE.
Mantau, M. J., Berkenbrock, C. D. M., and Berkenbrock, G. R. (2017). Analise de requisitos de percepção em um groupware móvel síncrono. Revista de Sistemas de Informação da FSMA, 1(19):16–33.
Nickerson, R. C., Varshney, U., and Muntermann, J. (2013). A method for taxonomy development and its application in information systems. European Journal of Information Systems, 22(3):336–359.
Niemantsverdriet, K., Essen, H. V., Pakanen, M., and Eggen, B. (2019). Designing for awareness in interactions with shared systems: The dass framework. ACM Transaction on Computer-Human Interaction, 26(6):1–41.
Pasquali, L. (2020). TRI–Teoria de resposta ao item: Teoria, procedimentos e aplicações. Editora Appris.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.
Seebach, C., Beck, R., and Pahlke, I. (2011). Situation awareness through social collaboration platforms in distributed work environments. In International Conference on Information Systems 2011, ICIS 2011, volume 1, pages 1–22, Shanghai, China.
Strasser, A. (2017). Delphi method variants in information systems research: Taxonomy development and application. Electronic Journal of Business Research Methods, 15(2):pp120–133.
Tenenberg, J., Roth, W.-M., and Socha, D. (2016). From i-awareness to we-awareness in cscw. Computer Supported Cooperative Work (CSCW), 25(4):235–278.
Van Solingen, R. and Berghout, E. W. (1999). The Goal/Question/Metric Method: a practical guide for quality improvement of software development. McGraw-Hill.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a repli cation in software engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE ’14, pages 1–10, New York, NY, USA. Association for Computing Machinery.
Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. (2012). Experimentation in software engineering. Springer Science & Business Media.
Antunes, P., Herskovic, V., Ochoa, S. F., and Pino, J. A. (2014). Reviewing the quality of awareness support in collaborative applications. Journal of Systems and Software, 89(1):146–169.
Bailey, K. D. (1994). Typologies and taxonomies: An introduction to classification techniques, volume 102. Sage.
Baker, F. B. (2001). The basics of item response theory. ERIC.
Baker, F. B. and Kim, S.-H. (2004). Item response theory: Parameter estimation techniques. CRC press.
Baker, F. B. and Kim, S.-H. (2017). The basics of item response theory using R. Springer.
Basili, V. (1992). Software modeling and measurement: The goal/question/metric paradigm. Technical Report.
Beecham, S., Hall, T., Britton, C., Cottee, M., and Rainer, A. (2005). Using an expert panel to validate a requirements process improvement model. Journal of Systems and Software, 76(3):251–275.
Collazos, C. A., Gutierrez, F. L., Gallardo, J., Ortega, M., Fardoun, H. M., and Molina, A. I. (2019). Descriptive theory of awareness for groupware development. Journal of Ambient Intelligence and Humanized Computing, 10(12):4789–4818.
de Andrade, D. F., Tavares, H. R., and da Cunha Valle, R. (2000). Teoria da resposta ao item: conceitos e aplicações. ABE, Sao Paulo.
DeVellis, R. F. (2016). Scale development: Theory and applications, volume 26 of Applied Social Research Methods 26. Sage Publications, Inc, 4th edition.
Dourish, P. and Bellotti, V. (1992). Awareness and coordination in shared workspaces. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work, CSCW ’92, pages 107–114, New York, NY, USA. Association for Computing Machinery
Gallardo, J., Bravo, C., and Molina, A. I. (2018). A framework for the descriptive specification of awareness support in multimodal user interfaces for collaborative activities. Journal on Multimodal User Interfaces, 12(2):145–159.
Gross, T. (2013). Supporting effortless coordination: 25 years of awareness research. Computer Supported Cooperative Work, 22(4-6):425–474.
Hinkelmann, K. and Kempthorne, O. (2005). Design and analysis of experiments: Advanced experimental design. Wiley-Interscience.
Hummel, D., Schacht, S., and Maedche, A. (2016). Determinants of multi-channel behavior: Exploring avenues for future research in the services industry. In ICIS.
Khalilijafarabad, A., Helfert, M., and Ge, M. (2016). Developing a data quality research taxonomy-an organizational perspective. In ICIQ, pages 176–186.
Mantau, M. J. and Benitti, F. B. V. (2022a). Awareness support in collaborative system: Reviewing last 10 years of cscw research. In 5th IEEE International Conference on Computer Supported Cooperative Work in Design, CSCWD 2022, Hangzhou, China, May 4-6, 2022, pages 564–569. IEEE.
Mantau, M. J. and Benitti, F. B. V. (2022b). Towards an awarenes taxonomy. In 25th IEEE International Conference on Computer Supported Cooperative Work in Design, CSCWD 2022, Hangzhou, China, May 4-6, 2022, pages 495–500. IEEE.
Mantau, M. J., Berkenbrock, C. D. M., and Berkenbrock, G. R. (2017). Analise de requisitos de percepção em um groupware móvel síncrono. Revista de Sistemas de Informação da FSMA, 1(19):16–33.
Nickerson, R. C., Varshney, U., and Muntermann, J. (2013). A method for taxonomy development and its application in information systems. European Journal of Information Systems, 22(3):336–359.
Niemantsverdriet, K., Essen, H. V., Pakanen, M., and Eggen, B. (2019). Designing for awareness in interactions with shared systems: The dass framework. ACM Transaction on Computer-Human Interaction, 26(6):1–41.
Pasquali, L. (2020). TRI–Teoria de resposta ao item: Teoria, procedimentos e aplicações. Editora Appris.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.
Seebach, C., Beck, R., and Pahlke, I. (2011). Situation awareness through social collaboration platforms in distributed work environments. In International Conference on Information Systems 2011, ICIS 2011, volume 1, pages 1–22, Shanghai, China.
Strasser, A. (2017). Delphi method variants in information systems research: Taxonomy development and application. Electronic Journal of Business Research Methods, 15(2):pp120–133.
Tenenberg, J., Roth, W.-M., and Socha, D. (2016). From i-awareness to we-awareness in cscw. Computer Supported Cooperative Work (CSCW), 25(4):235–278.
Van Solingen, R. and Berghout, E. W. (1999). The Goal/Question/Metric Method: a practical guide for quality improvement of software development. McGraw-Hill.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a repli cation in software engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE ’14, pages 1–10, New York, NY, USA. Association for Computing Machinery.
Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. (2012). Experimentation in software engineering. Springer Science & Business Media.
Publicado
22/05/2023
Como Citar
MANTAU, Márcio J.; BENITTI, Fabiane B. V..
The Awareness Assessment Model: measuring the awareness and collaboration support over the participant’s perspective. In: SIMPÓSIO BRASILEIRO DE SISTEMAS COLABORATIVOS (SBSC), 18. , 2023, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 30-43.
ISSN 2326-2842.
DOI: https://doi.org/10.5753/sbsc.2023.229064.
