Automated Assessment of Creativity of Mobile Apps in the Educational Context

• Nathalia da Cruz Alves Federal University of Santa Catarina http://orcid.org/0000-0002-3523-9199
• Christiane Gresse von Wangenheim Federal University of Santa Catarina https://orcid.org/0000-0002-6566-1606

DOI: https://doi.org/10.5753/educomp.2024.237500

Abstract

Creativity is an important 21st century skill that can be developed as part of computing education. One of the ways to foster creativity is through teaching the development of computational artifacts, such as mobile applications. Although there are several models for measuring creativity, evaluating the creativity of mobile applications in computing education remains relatively unexplored, with most existing models relying on manual assessment by humans. Although human assessment is important, it does not always cover all relevant aspects and can be susceptible to personal biases, preferences, and knowledge. Therefore, this article presents an analytical and automated model to evaluate the creativity of mobile applications. According to the definition of creativity, the model assesses originality, flexibility, and fluency. Statistical analysis results indicate the reliability and validity of the model. These results aim to contribute to the advancement of creativity assessment in computing education through a consistent assessment model, which can be complemented with human assessment, allowing a holistic assessment benefiting both educators and students.

Keywords: Creativity, Assessment, Mobile application, Computing education

References

Nathalia da Cruz Alves. Assessing the Creativity of Mobile Applications in Computing Education. 2023. Tese (Doutorado em Ciência da Computação) – PPGCC/Universidade Federal de Santa Catarina. [link].

Nathalia da Cruz Alves, Christiane. Gresse von Wangenheim, Lúcia Helena Martins-Pacheco (2021). Assessing Product Creativity in Computing Education: A Systematic Mapping Study. Informatics in Education, 20(1), 19-45. DOI: 10.15388/infedu.2021.02.

Teresa M. Amabile. 1996. Creativity in Context. Westview Press, Boulder, CO.

Victor R. Basili, Gianluigi Caldiera, H. Dieter Rombach. 1994. The Goal Question Metric Approach. In Encyclopedia of Software Engineering. Wiley.

Satabdi Basu. 2019. Using Rubrics Integrating Design and Coding to Assess Middle School Students' Open-ended Block-based Programming Projects. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, 1211–1217. DOI: 10.1145/3287324.3287412.

Roger E. Beaty, Dan R. Johnson. 2021. Automating creativity assessment with SemDis: An open platform for computing semantic distance. Behavior Research 53, 2, 757–780. DOI: 10.3758/s13428-020-01453-w.

R. A. Beghetto. 2010. Creativity in the classroom. In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge Handbook of Creativity (pp. 447-463). New York: Cambridge University Press.

Vicki Bennett, Kyu Han Koh, Alexander Repenning. 2013. Computing creativity: Divergence in computational thinking. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education. ACM, 359-364. DOI: 10.1145/2445196.2445302.

Timothy A. Brown. 2006. Confirmatory factor analysis for applied research. New York.

David Cavallo, Helena Singer, Alex S. Gomes, Ig I. Bittencourt, Ismar F. Silveira. 2016. Inovação e Criatividade na Educação Básica: Dos conceitos ao ecossistema. Revista Brasileira de Informática na Educação, 24(2).

Sally A. Fincher, Anthony V. Robins (Eds.). 2019. The Cambridge Handbook of Computing Education Research. Cambridge: University Press.

David B. Flora. 2020. Your Coefficient Alpha Is Probably Wrong, but Which Coefficient Omega Is Right? A Tutorial on Using R to Obtain Better Reliability Estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484-501. DOI: 10.1177/2515245920951747.

Lilach Gal, Arnon Hershkovitz, Andoni Eguíluz, Mariluz Guenaga, Pablo Garaizar. 2017. Suggesting a Log-Based Creativity Measurement for Online Programming Learning Environment. In Proceedings of the 4th Conference on Learning at Scale, 273-277. DOI: 10.1145/3051457.3054003.

Louis W. Glorfeld. 1995. An improvement on Horn's parallel analysis methodology for selecting the correct number of factors to retain. Educational and Psychological Measurement, 55(3), 377-393.

Joy Paul Guilford. 1950. Creativity. American Psychologist, 5, 444-454. DOI: 10.1037/h0063487.

Joseph F. Hair, et al. 2009. Multivariate data analysis (7th ed.). Prentice Hall.

Andrew F. Hayes, Jacob J. Coutts. 2020. Use Omega Rather than Cronbach’s Alpha for Estimating Reliability. But…, Communication Methods and Measures, 14, 1, 1-24, DOI: 10.1080/19312458.2020.1718629.

Arnon Hershkovitz, Raquel Sitman, Rotem Israel-Fishelson, Andoni Eguíluz, Pablo Garaizar, Mariluz Guenaga. 2019. Creativity in the acquisition of computational thinking. Interactive Learning Environments 27, 6 (Sep. 2019), 813-829. DOI: 10.1080/10494820.2019.1610451.

James C. Kaufman. 2012. Counting the muses: Development of the Kaufman Domains of Creativity Scale (K-DOCS). Psychology of Aesthetics, Creativity, and the Arts, v. 6, n. 4, p. 298–308. DOI: 10.1037/a0029751.

James C. Kaufman, Ronald A. Beghetto. 2009. Beyond Big and Little: The Four C Model of Creativity. Review of General Psychology 13, 1 (Mar. 2009), 1–12. DOI: 10.1037/a0013688.

James C. Kaufman, Jonathan A. Plucker, John Baer. 2008. Essentials of Creativity Assessment. John Wiley & Sons.

Prapti Khawas, Peeratham Techapalokul, Eli Tilevich. 2019. Unmixing Remixes: The How and Why of Not Starting Projects from Scratch. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, 169-173. Memphis, TN, USA. DOI: 10.1109/VLHCC.2019.8818834.

Kyu Han Koh. 2011. Computing indicators of creativity. In 2011 IEEE Symposium on Visual Languages and Human-Centric Computing. DOI: 10.1109/vlhcc.2011.6070407.

Anastasia Kovalkov, Benjamin Paaßen, Avi Segal, Niels Pinkwart, Kobi Gal. 2021. Automatic Creativity Measurement in Scratch Programs Across Modalities. IEEE Transactions on Learning Technologies 14, 6 (Dec. 2021), 740-753. DOI: 10.1109/TLT.2022.3144442.

Anastasia Kovalkov, Avi Segal, Kobi Gal. 2020. Inferring Creativity in Visual Programming Environments. In Proceedings of the Seventh ACM Conference on Learning @ Scale. Association for Computing Machinery, New York, NY, USA, 269–272. DOI: 10.1145/3386527.3406725.

Jiwen Luo, Feng Lu, Tao Wang. 2020. A Multi-Dimensional Assessment Model and Its Application in E-learning Courses of Computer Science. In Proceedings of the 21st Annual Conference on Information Technology Education, 187-193. ACM, New York, NY, USA. DOI: 10.1145/3368308.3415388.

Sven Manske, Ulrich Hoppe. 2014. Automated Indicators to Assess the Creativity of Solutions to Programming Exercises. In Proceedings of the IEEE 14th International Conference on Advanced Learning Technologies, 497-501. DOI: 10.1109/ICALT.2014.147.

MIT App Inventor. 2023. App of the Month. [link].

Eni Mustafaraj, Franklyn A. Turbak, Maja Svanberg. 2017. Identifying Original Projects in App Inventor. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference, 567-572. Florida: Association for the Advancement of Artificial Intelligence. URL: [link].

John D. Patterson, Baptiste Barbot, James Lloyd-Cox, Roger Beaty. 2022. AuDrA: An automated drawing assessment platform for evaluating creativity. PsyArXiv. DOI: 10.31234/osf.io/t63dm.

Jonathan Plucker, Ronald A. Beghetto, Gayle Dow. 2004. Why isn't creativity more important to educational psychologists? Potential, pitfalls, and future directions in creativity research. Educational Psychologist, 39, 83–96. DOI: 10.1207/s15326985ep3902_1.

Mel Rhodes. 1961. An Analysis of Creativity. The Phi Delta Kappan, 42(7), 305-310.

Graeme D. Ritchie. 2001. Assessing Creativity. In Proceedings of the AISB symposium on AI and creativity in arts and science (pp. 3-11). York: The Society for the Study of Artificial Intelligence and Simulation of Behaviour.

Valerie J. Shute, Chen Sun, Jodi Asbell-Clarke. 2017. Demystifying computational thinking. Educational Research Review, 22, 142-158. DOI: 10.1016/j.edurev.2017.09.003.

Taguma, M. F. (2018). Future of Education and Skills 2030: Conceptual Learning Framework. OECD. Retrieved August 4, 2021 from [link].

Ellis Paul Torrance. 2008. The Torrance Tests of Creative Thinking Norms—Technical Manual Figural (Streamlined) Forms A & B. Bensenville: IL: Scholastic Testing Service.

Franklyn A. Turbak, Eni Mustafaraj, Maja Svanberg, Michael Dawson. 2017. Work in Progress: Identifying and Analyzing Original Projects in an Open-Ended Blocks Programming Environment. In Proceedings of the 23rd International Conference on Distributed Multimedia Systems, Visual Languages and Sentient Systems, 115-117. Florida: Association for the Advancement of Artificial Intelligence.

Universidade Federal de Santa Catarina - Computação na Escola. CodeMaster. [link]

Joke Voogt, Natalie Pareja Roblin. 2012. A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies 44, 3 (May 2012), 299–321. DOI: 10.1080/00220272.2012.668938.