UtopicSense: a tool to support the use of synesthesia as an assistive resource in 2D games
Resumo
The concept of synesthesia in games comprises providing stimuli in more than one sense, such as displaying visual effects concurrently with sound effects on a screen. Algorithmic synesthesia is a concept that describes multimedia processes in which sounds, images, and in some cases, haptic sensations have as generators the same computational process or data source. Game developers can leverage such concept as an ancillary resource for people with hearing loss or deafness. Despite its advantages to the gaming experience, synesthesia is rarely contemplated in gaming design with the focus on accessibility. This situation usually happens because developers are unaware of its benefits or want to avoid additional costs and more implementation work on the project's schedule. This paper presents UtopicSense, a tool that automates implementing synesthesia in 2D games inside Unity. To create it, we identified positive and negative accessibility aspects perceived through the analysis of 2D games. We then developed the tool which allows users to configure aspects of the generated visual effects through a Unity custom editor panel. We evaluated the tool in terms of usability with potential users (Unity game developers). They provided feedback and evaluated the tool in terms of ease of use, flexibility, and usefulness. Participants expressed positive reactions towards UtopicSense and indicated that it fulfilled its goal of integrating algorithmic synesthesia into 2D games in an easier way.
Referências
K. Bierre, M. Hinn, T. Martin, M. McIntosh, T. Snider, K. Stone, and T. Westin, “Accessibility in games: Motivations and approaches,” Whitepaper, International Game Developers Association (IGDA), 2004.
A. S. Bastos, R. F. Gomes, C. C. dos Santos, and J. G. R. Maia, “Synesthesia: A study on immersive features of electronic games,” SBC Journal on Interactive Systems, vol. 9, no. 2, pp. 38–51, 2018.
F. R. S. Coutinho, “Revisiting game accessibility for deaf and hard of hearing players,” M.S. thesis, UFMG, Belo Horizonte, MG, Brazil, 2012.
C. L. Siying Liang, “Synesthesia and its implications on video game de-sign,”Design Engineering, pp. 995 – 1002, Dec. 2020. [Online]. Available: http://thedesignengineering.com/index.php/DE/article/view/1136
GameDesign,“The top 10 videogame engines,” https://www.gamedesigning.org/career/video-game-engines/ (accessed Dec. 03, 2018).
F.Tanant, “The top 10 videogame engines,” https://www.websitetooltester.com/en/blog/best-game-engine/ (accessed Dec. 03, 2018).
Unity, “Games made with unity,” https://unity3d.com/pt/games-made-with-unity (accessed Dec. 03, 2018).
S. R. Basbaum, “Synesthesia and digital perception,” 3rd Subtle Technologies Festival, Toronto, 2003, pp. 01–20.
S. R. Basbaum, Sinestesia, arte e tecnologia. São Paulo, Brazil: Fapesp-Annablume, 2002.
R. T. Dean, M. Whitelaw, H. Smith, and D. Worrall, “The mirage of real-time algorithmic synaesthesia: Some compositional mechanisms and research agendas in computer music and sonification,” Contemporary Music Review, vol. 25, no. 4, pp. 311–326, 2006.
N. Sagiv, R. T. Dean, and F. Bailes, Algorithmic synesthesia, in R. T. Dean, Ed., The Oxford Handbook of Computer Music. Oxford, UK: Oxford Univ. Press, 2009, pp. 294-311.
A. S. Bastos, R. F. Gomes, C. C. dos Santos, and J. G. R. Maia,“ Synesthesia: A study on immersive features of electronic games,” SBCJournal on Interactive Systems, vol. 9, no. 2, pp. 38–51, 2018.
Journey. (2012) .Thatgamecompany. [Online]. Available: http://thatgamecompany.com/journey/
J. M. E. d. Silva, A. D. Castro Callado, and P. M. Jucá, “Representing sentiment using colors and particles to provide accessibility for deaf and hard of hearing players,” 2018 17th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames), 2018, pp. 221-22109, doi: 10.1109/SBGAMES.2018.00034.
F. Coutinho, R. O. Prates, and L. Chaimowicz, “An analysis of information conveyed through audio in an FPS game and its impact on deaf players experience,” in 2011 Brazilian Symposium on Games and DigitalEntertainment. IEEE, 2011, pp. 53–62.
The Legend of Zelda: link to the Past. (1991). Nintendo. [SNES Cartridge].
Limbo. (2010). Playdead. [Online]. Available: http://playdead.com/games/limbo
Mark of the Ninja. (2012). Klei Entertainment. [Online]. Available: https://www.klei.com/games/mark-ninja.
Ori and the Blind Forest. (2015). Moon Studios. [Online]. Available: http://www.oriblindforest.com/
Rayman Origins. (2011). Ubisoft. [Online]. Available: https://www.ubisoft.com/en-gb/game/rayman/origins
Rayman Legends. (2013). Ubisoft. [Online]. Available: https://www.ubisoft.com/pt-br/game/rayman/legends
Shadow Dancer. (1989). Sega. [Sega Genesis Cartridge].
Sonic Mania. (2017). Sega. [Online]. Available: https://www.sega.com/games/sonicmania
Sonic the Hedgehog. (1991). Sega. [Sega Genesis Cartridge].
Sonic the Hedgehog 2. (1992). Sega. [Sega Genesis Cartridge].
Streets of Rage. (1991). Sega. [Sega Genesis Cartridge].
B. Yuan, E. Folmer, and F. C. Harris, “Game accessibility: a survey,” Universal Access in the Information Society, vol. 10, no. 1, pp. 81–100, 2011.
D. N. Nogueira, F. R. Coutinho, W. Soares Jr, R. O. Prates, and L. Chaimowicz, “Analyzing the use of sounds in FPS games and its impact for hearing impaired users,” Proceedings of SBGames 2012, SBC, 2012, pp.127–133.
R. da Silva Lima, “Da nota ao som: explorando territórios harmônicos,” M.S.. thesis, Instituto de Artes, Universidade Estadual de Campinas (UNICAMP), Campinas, SP, Brazil, 2009.
F. J. Harris, “On the use of windows for harmonic analysis with thediscrete fourier transform,” Proceedings of the IEEE, vol. 66, no. 1, pp.51–83, Jan.1978, doi: 10.1109/PROC.1978.10837.
J. S. Bendat and A. G. Piersol, Random Data: analysis and measurement procedures. John Wiley & Sons, 2011, vol. 729.
C. Roads and J. Strawn,The computer music tutorial. MIT press, 1996.
D. Hammershoi and H. Moller, “Sound transmission to and within the human ear canal,” The Journal of the Acoustical Society of America, vol. 100, no. 1, pp. 408–427, 1996.
S. Barbosa and B. Silva, Interação humano-computador. Elsevier Brasil, 2010.
