License.. Player Experience Evaluation: Which instrument should I

do Ceará | ticianne@virtual.ufc.br ] Abstract The influences, metrics, and applications of User Experience (UX) have been investigated in various contexts and is acknowledged as a driving force to promote game development choices. Recently, there has been a growing interest and need to explore the experience in the context of digital games, which require particular forms of Player Experience (PX) components due to their interaction. These particularities of digital games bring some specific models, characteristics and evaluation methods based on this field. Therefore, professionals and researchers must make conscious choices when planning these assessments. This research aims to provide an extension of a catalog of PX evaluation instruments, bringing analysis and discussions about the Player Experience and its concepts and also cataloging new instruments,


Introduction
The area of HumanComputer Interaction (HCI) has been broadly investigating User experience (UX) (Hassenzahl and Tractinsky, 2006; Nacke et al., 2019, as well as its appli cations, metrics, advancements, and influences in the inter action with many types of applications, including the in creasing area of digital games (Nacke et al., 2019). Digital games have become an important field in both Industry and Academy. Marketwise, the video games and esport sector topped U$118bn in 2018 and it keeps growing, with the ex pected annual growth rate of 5% 6% from 2018 to 2023, being one of the most preeminent cultural industries of me dia entertainment (Van Eeden and Chow, 2019). The fact that successful games have the ability to engage users for hours and make them learn complex tasks has instigated the interest of the academic community around game features and game experience particularities (Malone, 1982; Carter et al., 2014; Bernhaupt et al., 2015. Digital games and HCI have been linked since the first CHI conference in 1982, when Malone, based on his study on computer games, reinforced a set of design principles that could be applied for "enjoyable" user interfaces (Malone, 1982). Since then, researchers have seen that we (as HCI aca demics) could learn from games, but we could also support the game development industry and that's where Game User Research (GUR) takes place. Seif ElNasr et al. (2012) in troduce GUR as "a field concerned with developing a set of techniques and tools to measure the users' behaviors and ulti mately improve their experiences as they engage with games" (Seif ElNasr et al., 2012).
The UX is one of the driving forces for game designers when making choices during the project and development of games. This evidence was first identified in the work of De Anda and Kocurek (2016), after reviewing three books com monly used in game design courses: The Art of Game De sign: A Book of Lenses (Schell, 2014); Challenges for Game Designers: NonDigital Exercises for Video Game Designers (Brathwaite and Schreiber, 2008); and Game Design Work shop: A Playcentric Approach to Creating Innovative Games (Fullerton, 2014). To emphasize the importance of UX for game design, the authors state that designing a game is re lated to creating the best experience possible for the play ers. This process occurs by incorporating practices that go beyond programming to cover iterative design, game test ing and attention to User Experience (DeAnda and Kocurek, 2016). Their viewpoint is in accordance with the earlier HCI perspective brought by Bernhaupt (2015), which sees the main goal of developing a game as creating a product that is fun to play, has surprises, provides challenges to players and promotes social connections. In HCI, the particular forms of interactivity of digital games is what divides them from other paradigms of interactive digital systems, such as desktop sys tems, that are developed to execute a specific group of tasks.
Thus, digital games demand particular ways of evaluating the experience of players (Sánchez et al., 2012), which moti vated the development of several players' experience evalu ation approaches that have been used during the game devel opment and also after the game release (Bernhaupt, 2015). Over the years, several Player Experience (PX) evaluation instruments and guidelines were either developed or adapted specifically for games (Sánchez et al., 2012), as PX evalua tion towards gaming in Industry has been carried out since before GUR became an established research domain. How ever, these evaluations and often, the employed instruments were usually done informally and without proper guidelines (Wiemeyer et al., 2016).
Besides, as research in games interaction and development advanced, several different terms arose to somehow describe the experience in games (e.g., Gaming Experience, Game Ex perience, Player Experience, User Experience). These terms are commonly used without a proper discussion of their defi nition and meaning, although they usually carry different per spectives and understandings (Sánchez et al., 2012). In this context, evaluating the experience of players in digital games is a rather complicated task, due to the inherent complexity of games in addition to the several different ways of address ing Player Experience, the wide variety of evaluation instru ments, and the uncertainty about the assumptions on which they are built.
This scenario is even more difficult in countries where the Games Industry is mainly composed of independent game developers that generally work with a limited budget com monly based in crowdfunding and rely on small teams in which one person exercises different functions (e.g., Brazil) (Costanti, 2018). In these cases, it is uncommon to find a team with an HCI expert to consider the multiple human fac tors and experience components and then choose the most ap propriate ways to evaluate a game under development. Con sequently, at times, evaluations are planned and conducted based on the game developer's personal experiences and re stricted knowledge about available methods and instruments, which compromises the quality of players' experience evalu ation.
This paper aims to help to fill in the gap of lacking infor mation about instruments to support the evaluation of Player Experience in digital games and their assumptions, consid ering the different components of the PX and types of avail able instruments. This work is an extension of a previously published paper  and aims to provide deeper analysis and discussions about what the PX evalua tion instruments measure, their applications in different con texts and about the terms used to define the Player Experi ence. In this paper, despite the lack of consensus about the terms used to describe the experience in games, we adopt the term Player Experience to present our discussions and analysis. The present study is also able to add to the previ ous work by presenting an upgraded version of an interac tive catalog composed of 58 instruments for evaluating mul tiple aspects of the PX in digital games and virtual environ ments. The upgrade consisted of the cataloging of 13 more instruments and executing additional analysis of all of them to expand the available data on each one (language, expe rience perspective, type of collected data, and evaluated di mensions). Moreover, we discuss different perspectives for understanding the Player Experience, which are the basis for the construction and use of the instruments. Finally, we also discuss how the cataloged instruments address these differ ent perspectives, as well as some trends and issues for the GUR field.
We expect this paper to help game developers and design ers, UX and PX researchers, and students of corelated areas to make informed choices when planning the evaluation of the Player Experience in digital games, as well as to outline future research in this field.

Experience in Games
To better understand the panorama of Player Experience per spectives in games evaluation, in this section, we discuss the different terms describing such views. Then, we discuss the differences between Playability and Player Experience. Lastly, we explore some of the fundamentals behind the Player Experience and its components and dimensions.

Multiple Terms and perspectives
Different perspectives affecting both game design and eval uation have been discussed in the literature for understand ing UX in games. Distinct terms have been adopted to de scribe these viewpoints in the literature concerning UX eval uation in digital games, such as Game Experience (Poels et al., 2007a; Lai et al., 2012, Gaming experience (Calvillo Gámez et al., 2015; Jennett et al., 2008, Player Experience (Lazzaro, 2008a; Wiemeyer et al., 2016, and User Experi ence (Qin et al., 2009; Sweetser and. How ever, these terms are frequently used without a clear distinc tion of their definitions and what they represent to the studies (Wiemeyer et al., 2016). Poels et al. (2007a) described the term Game Experience as a multidimensional and multilayered concept that refers to the users' feelings and experiences when playing digital games. In their study, the authors explored this concept in fo cus groups. The results allowed the categorization of aspects that would constitute Game Experience: enjoyment, flow, imaginative immersion, sensory immersion, suspense, com petence, negative affect, control, and social presence. CalvilloGámez et al. (2015) refer to the term Gaming Ex perience when they presented the Core Elements of Gaming Experience (CEGE). CEGE is where a positive experience or enjoyment is achieved according to the elements defined as Videogame and Puppetry. For them, Videogame is re lated to the player's interaction, while Puppetry is related to the player's perception of the game.
As for Player Experience (PX), Wiemeyer et al. (2016) de picted PX as the quality of playergame interactions, and it is typically investigated during and after the interaction with games. In this definition, PX is also divided into three lev els: the psychological (social) level, which refers to the in dividual experience, the behavioral level and the physiolog ical level. This distinction allows the experience to be eval uated more precisely by integrating physiological methods (e.g., heart rate, electrodermal activity) and behavioral meth ods (e.g., eyetracking) to supplement the commonly used psychological approaches (e.g., surveys and questionnaires) (Wiemeyer et al., 2016).
User Experience is a broader term that is also used to address games evaluation and has been widely investigated within the HCI field. According to the definition in ISO 9241 11, User Experience encompasses "user's perceptions and re sponses that result from the use or anticipated use of a sys tem, product or service" (Iso, 2018). However, literature re views and surveys indicate that there is no agreement about the scope and definition of UX in both Academy and Indus try (Law et al., 2009; Melo and. The same phe nomenon is seen in the context of games (Bernhaupt, 2015).
Some authors view UX as a construct that should be an in trinsic part of the game development lifecycle, in which prac titioners should use specific kinds of UX evaluation methods (Bernhaupt, 2015). In this perspective, Bernhaupt (2015) dis cusses that while user experience evaluation methods from HCI are used during game development, HCI as a field is borrowing and exploring aspects of the gaming experience like immersion, fun, and flow to better understand the con cept of user experience.
Some researchers are focused on distinguishing the terms addressing UX in games. Lazzaro (2008b) argues that UX and PX are different concepts: UX would be the experience of game use, while PX is related to which kind of enjoyment the player is seeking. In Lazzaro perspective, PX analyzes what keeps the player away from having fun, while UX ob serves what creates boundaries to the ability of gaming. On the other hand, Nacke and Drachen (2011) consider PX as UX in the specific context of digital games.
Literature has also compared Game Experience and Player Experience. Wiemeyer et al. (2016) argue that Game Experi ence had its place taken by PX in a similar way that usability had its place taken by UX. In this perspective, Game Experi ence is closer to technology than to the subjective experience of humans (Lazzaro, 2008b). Hence, for the authors, Player Experience is a more appropriate term than Game Experi ence, as the one having this specific experience is the player (Wiemeyer et al., 2016).
The choice of a term that best describes the experience in games is so far an open debate. Among the existing terms for describing experience in the context of digital games, in this study, we chose to address experience in games as Player Experience (PX) following Wiemeyer et al. rationale.

Playability and Player Experience
Despite various perspectives to define the experience in dig ital games, there is a general agreement that usability is es sential, but is not enough or determinant in game develop ment (Nacke and Drachen, 2011), due to its standard metrics are not mapped directly to game evaluation (e.g., effective ness measured as task completion or efficiency, error rates) (Wiemeyer et al., 2016). Game design requires a primary fo cus on human and subjective factors, such as the emotional and cultural aspects of the players (Sánchez et al., 2012; Wiemeyer et al., 2016. To measure and evaluate usability within game develop ment, researchers need to combine classical usability fac tors with the subjective aspects inherent in digital games (Sánchez et al., 2012). Thus, the concept of Playability was coined. According to Sánchez et al. (2012) this term mea sures and describes the quality of a game at a technological level (e.g., within the scope of rules, mechanics, design, and goals) and is affected by factors like graphics, sounds, story line, and control.
It is common to confuse Playability with Player Experi ence, but the terms include aspects that are quite distinct when analyzed. In a nutshell, Playability seeks to guarantee a good experience at a technological level, whereas Player Experience is about the quality of playergame interactions during and after they occur (Wiemeyer et al., 2016). PX fo cuses on the player and is based on the measurement of three levels of experience: sociopsychological aspects, be havioral and physiological reactions (Wiemeyer et al., 2016). Hence, Playability is the basis for a good Player Experience

Player Experience Components and Di mensions
According to Wiemeyer et al., one must consider a diversity of factors to comprehend what Player Experience is. Those PX factors are the elements that contribute to this type of experience and come from many disciplines (e.g., neuro physiology, psychology, and sociology). Although there is no consensus on what specific factors constitute PX (Nacke and Drachen, 2011), there is a shared comprehension that PX is a multidimensional and multilayered construct (Poels et al., 2007b). Thus, several psychophysiological models have been developed to explain PX's structure and the di verse components influencing this experience (Wiemeyer et al., 2016). A variety of terms have been used to name what Wiemeyer et al. (2016) called factors, such as dimensions, constructs, and components. However, there is no clear distinction be tween these terms within the literature. For example, Poels et al. (2007a) and Poels et al. (2012) consider Flow and Im mersion factors as dimensions, while Denisova et al. (2016) and Ermi and Mäyrä (2005) refer to them as components. There are also studies where the terms dimensions and com ponents are used interchangeably, without definition of their correlation (Wiemeyer et al., 2016) (Drachen et al., 2010. In this paper, we chose to use the terms components and dimensions to describe PX factors. We consider components as the factors that manifest different facets of the Player Ex perience (e.g., Flow, Immersion and Presence); and dimen sions as the elements that scope components (e.g., engage ment, engrossment and total immersion are dimensions of the PX component Immersion ). A PX com ponent may be described by different dimensions, depending on the author's theoretical assumptions. For example, Poels et al. (2007a) differs from Cheng et al. definition of immer sion and considers that it is made up of sensory immersion and imaginative immersion.
Hence, in this paper, we consider PX as a construct that characterize the quality of the playergame interaction in terms of a set of components which may be defined by a subset of dimensions, encompassing sociopsychological as pects, and behavioral and physiological reaction.
The variety of understandings about the same components results in different approaches of PX evaluation. This phe nomenon is clear when considering some of the most usual components of PX: Immersion (Jennett et al., 2008; Cheng et al., 2015, Enjoyment (Fitzgerald et al., 2020; Sweetser and Immersion is usually addressed as the outcome of a good experience (Jennett et al., 2008), and it is used to measure the degree of involvement with a game. Jennett et al. (2008) de veloped a selfreport questionnaire in which the dimensions of immersion are: cognitive involvement, realworld disso ciation, emotional involvement, challenge and control. How ever the Game Immersion Questionnaire (GIQ) , which evaluates the same PX component, describes it with different dimensions: engagement, engrossment, and to tal immersion.
Another example can be seen in Enjoyment, which can be defined as the feeling of pleasure resultant from gaming , and is the most important goal in digital games as it determines whether the user is willing to play the game (Sweetser and Wyeth, 2005). On one hand, the Exergame Enjoyment Questionnaire (EEQ) , consider immersion (here understood as a dimension instead of a PX component), intrinsically rewarding activity, control, and exercise as dimensions of Enjoyment. On the other hand, the EGameFlow Scale (Fu et al., 2009) considers concentration, goal clarity, feedback, challenge, autonomy, immersion, social interaction, and knowledge improvement as Enjoyment dimensions. We highlight that these PX components as well as other PX components may also have slightly different definitions and dimensions from one measurement instrument to another. Nevertheless, each different perspective brought by distinct evaluation perspectives contributes to analyzing PX in games and virtual environments more thoroughly.

Methodology
This work is an extension of a previous work describing the PX Instruments Catalog , in which we analyzed and cataloged 47 instruments for evaluating differ ent components of experience in games and virtual environ ments, based on four attributes (type of instrument, target users, UX qualities evaluated and year of publication).
The present study aims to refine, expand, and deepen the analysis and discussions produced in the initial research. Hence, we searched more instruments in the literature, re viewed the instrument papers, gathered more information about each of them and analyzed the data of the final 58 in struments according to eight attributes (Table 1).
Our methodology followed four steps ( Figure 1): 1) Liter ature search, 2) Refinement and expansion of PX instrument catalog, 3) Data extraction, 4) Data analysis and categoriza tion of instruments.
First, a literature search was conducted to deepen the theo retical background on PX fundamentals. This step fomented a broader understanding of the different terms describing the experience in digital games, (including Game Experience, Player Experience and User Experience), the differences be tween playability and Player Experience, in addition to dis cussions about PX components and dimensions. This step was important to define the attributes that would later be used in data analysis (as described in Step 3). In Step 2, aiming to refine the PX Instruments Catalog , two researchers reviewed the extracted data of the 47 previously cataloged instruments. Each re searcher read the papers, doublechecking and supplement ing information on type of instrument, approach, PX com ponents, and target users. Researchers also identified and re moved two duplicated instruments, which were described in different papers. Then, to expand the PX Instruments Cat alog, we identified 13 new PX instruments after running a forward snowballing (Wohlin, 2014) on the 45 papers on the PX Instruments Catalog, resulting on 58 papers. In Step 3, a researcher read the full text and extracted data from the 58 papers. In addition to the original set of four at tributes, he analyzed four additional attributes for each instru ment, resulting in the final eight: 1. type of instrument (e.g., scales and questionnaires, soft wares and equipments, twodimensional diagram); 2. type of approach (e.g., qualitative, quantitative, quali quantitative); 3. PX components; 4. dimensions describing the PX components; 5. target users; 6. instrument language; 7. perspective of experience (i.e. terms authors used to re fer to experience in games); 8. type of collected data (i.e. the type of data the instru ments collect to evaluate the experience).
After that, another researcher reviewed the data extracted for each paper. In Step 4, two researchers analyzed the extracted data by tabulating and categorizing them according to the eight at tributes. We count the instruments in each type, in each com ponent and dimension and in each type of experience. After that, we used descriptive statistics to categorize and summa rize the data of the entire set of instruments and within each type of instrument. Besides, we also searched for trends in the instrument's data over the years and analyzed how their authors described the experience in games, their evaluated PX components and dimensions, as well as the relationships between them. The Table 1 shows the different attributes of the analysis in the previous paper  and the present study. The analysis of trends in the instruments data brought novel insights and perspectives about the definition of Player Experience, its facets, the understanding of its components and dimensions, the divergences about how to evaluate the PX components and about how cultural aspects are consid ered in PX evaluation. These topics are further discussed in Sections 4 and 5. Additionally, an updated version of the PX Instruments Catalog can be accessed in this address 1 .

Results
The Player Experience Instruments Catalog resultant from this research comprises 58 instruments that evaluate differ ent perspectives of experience in games and virtual environ ments (Table 11 and Table 12, in the Appendix). In this sec tion, we present the data of the instruments according to their types and attributes.

Overview
The 58 cataloged instruments evaluate 70 different compo nents of PX, which are showed in the Figure 2 (the size of the words is proportional to the number of instruments that evaluate the respective component). The components most evaluated by the instruments were immersion (evaluated by 11 instruments), presence (nine instruments) and challenge (seven instruments). We categorized the components evalu ated by two (3.45%) instruments as "Others", because their articles showed that the instruments also evaluated other as pects or constructs in addition to the Player Experience (Savi et al., 2011; Petri et al., 2016. The papers of the instruments presented a large amount of terms to define the PX compo nents and these terms diverge for each author. Therefore, it is important to highlight that this study's goal is not to analyze the theoretical reasoning behind them. We classified the 58 instruments into three different types: scales and questionnaires (82.76%), software and equipment (15.52%), and diagrams and twodimensional graph areas (1.72%). Table 2 exemplifies the instruments of each of those types and the components evaluated by them. As for target users, we identified three categories: children, learners and "players in general". The last one classifies in struments that do not determine a specific target user or are intended to all types of players. Only two (3.45%) out of 58 cataloged instruments are specifically targeted to children (Vissers et al., 2013; Moser et al., 2012 and also one (1.72%) is directed to learners (Fu et al., 2009), while 55 (94.83%) did not define a particular type of target player and/or were intended to all types of players.
The instruments that use the term "Player Experience" evaluate 31 different components. In comparison, those who use the term "Game Experience" evaluate 26 different com ponents, and the instruments developed with the perspective of "User Experience" evaluate 12 different components. Ta ble 3 shows all the perspectives of experience found, the num ber of instruments that use each one, and how many compo nents are evaluated by the instruments of each perspective.
The cataloged instruments were developed in different lan guages, so that 50 (86.21%) out of the 58 are in English only (e.g. Ravaja et al. (2004)), while five (8.62%) have an En glish version and a translated version (Dutch and Portuguese) (e.g. Petri et al. (2016)) and three (5.17%) instruments are in Portuguese only (e.g.VasconcelosRaposo et al. (2016)). Ta ble 4 shows the number of instruments which are not only in English and their references.

Instruments and components over the years
Over the years, we can observe the constancy with which new instruments are developed and also the prevalence of scales and questionnaires over other types of instrument. Since  1998 (when the oldest cataloged instrument was published ), at least one instrument for eval uation of experience in games was developed per year ex cept for the year 2000. Scales and questionnaires are the most recurrent type of instruments, so that every year since 1998, at least one instrument of this type was identified, except for the years 2000 and 2010 (Figure 3). Unlike scales and questionnaires, the publication of soft wares/equipments and twodimensional diagrams only oc curs years laters, from 2008 and 2013, respectively, and less frequently. Between 2008 and 2020, the softwares and equip ments rate per year is 0.69. From 2013 to 2020, the average of twodimensional diagrams is 0.13 per year. Meanwhile, the average of scales and questionnaires per year, from 1998 to 2020, is 2.09.
Although we observed a predominance of scales and ques tionnaires, the instruments of other types have been devel oped more frequently throughout the years. We identified 24 scales and questionnaires and only two instruments of other types developed from 1998 to 2009. Meanwhile, from 2010 to 2020, also 24 scales and questionnaires were developed, but we identified eight of other types (four times more than in the first period), which represents an increasing trend in the frequency of other types of instruments to evaluate the experience in digital games (Figure 4).
Regarding the components of the Player Experience, from 1998 to 2020, we noticed a significant increase in the number of evaluated PX components by the instruments throughout the years. Figure 5 shows the number of PX components mea sured by the instruments of each year. From 1998 to 2009, the instruments evaluated 26 different PX components, while be tween 2010 and 2020, 58 components of the experience were evaluated.

Types of Instruments
The different types of cataloged instruments present partic ular trends in their data. The data analysis showed differ ent concentrations of PX components, dimensions and tar get users between the scales and questionnaires and the other types of instruments.

Scales and Questionnaires
Among all types of cataloged instruments, verbal and nonver bal scales and questionnaires prevail with 48 (82.76%) instru ments, appearing significantly more than other types. Scales and questionnaires, despite their conceptual differences, are reported as a single category ("scale/questionnaire") because both terms are frequently used in an exchangeable way, alongside the cases in which scales are developed only for a specific questionnaire (e.g. Poels et al. (2007b)).
The components evaluated by scales and questionnaires are often constituted by different dimensions, according to their authors. Table 6 shows the dimensions considered in the most recurrent components evaluated by this type of in strument.
Regarding the target users of the scales and questionnaires, from the 48 cataloged scales and questionnaires, 46 (95.75%) are intended for all types of players, while only one (2.08%) was developed specifically for children (Moser et al., 2012) and also one (2.08%) focuses on learners (Fu et al., 2009).

Software, equipment, and twodimensional dia grams
Among the 58 cataloged instruments, nine (15.52%) are soft wares or equipments, representing the second most recurring type of instruments found. These nine instruments evaluate three different components (Table 7): Behavior (55.56%), followed by Emotion (33.33%), and Aesthetic experience (11.11%). All the instruments of this type evaluate the ex perience with all types of players. The other type of instrument we identified is two dimensional diagrams and graph areas, with only one instru ment, representing 1.72% of the total. The single instrument of this type intends to evaluate four different components (Ta ble 8), which are usability, challenge, the quantity of play, and general impression  and targets all types of players.

Components and dimensions
The cataloged instruments aim to evaluate different compo nents of the experience. In most instruments, these compo nents are fragmented in different dimensions that constitute them ( Figure 6). We found 93 different dimensions of the components of the experience. Eleven (11.83%) of these 93 dimensions are shared by more than one component (e.g. con trol is a dimension that describes the component Immersion and also the component Flow (Qin et al., 2009; Sweetser and. Table 9 shows: (i) these eleven dimensions; (ii) the components which they constitute; (iii) and the per centage of instruments which evaluate that component and consider the respective dimension.
Among the 93 dimensions found, only 18 (19.35%) appear in more than one instrument (e.g.control (Lin et al., 2002)) and 75 (80.65%) appear only once (e.g.tiredness ). The dimensions that appear more often in the instruments, constituting different PX components are chal lenge, control, and immersion. Challenge appears as a dimen sion that constitute four different PX components: Immer sion (Jennett et al., 2008), Flow (Lai et al., 2012), Enjoyment (Fu et al., 2009), and Gameful Experience (Högberg et al., 2019); Control is seen as part of four different PX compo nents: Enjoyment , Immersion (Qin et al., 2009), Playfulness , and Flow (Lai et al., 2012); Immersion which is itself addressed as a PX component is also used by some authors as a dimension that constitute other four PX components: Enjoyment , Engagement (Brockmyer et al., 2009), Gameful Experience (Högberg et al., 2019), and Flow (Sweetser and Wyeth, 2005). Table 10 shows examples of PX components and different dimensions found in the instruments that eval uate them.

Online Catalog of instruments
We organized and summarized the set of 58 instruments and its data in a virtual catalog, which is an updated version of the catalog presented by Borges et al. (2019). In its previ ous version, the catalog of PX instruments was integrated with the catalog of general UX instruments (Figure 7, in the Appendix). All instruments were sorted by the type of ap plication (e.g. Games and virtual environments, Hardware and robotics) and were displayed as a linear list without addi tional filters (Figure 8, in the Appendix). The navigation was problematic, especially for users who did not know which type of instruments they were looking for. When that was the case, the user would have to go through all the list in order to consult each instrument turning into a long and exhausting process.
In order to optimize the searching process, the PX evalua tion instruments were separated from the others. Also, three additional filters were added: type of instrument, targetuser, and PX components evaluated (Figure 9, in the Appendix). The new version can be accessed in the link available in this paper 2 .
The catalog structure was planned to help researchers and practitioners choose what instrument they should use to eval uate different components and dimensions of experience in games, based on their research goals.
Each instrument in the catalog presents the following in formation (as represented in Figure 10, in the Appendix): PX components, dimensions, type of instrument, type of ap proach, targetusers, reference and name, in addition to the instruments general procedure and the main idea.
The main idea and the general procedure present, respec tively, a brief description of what the instrument is, and how it should be administered in evaluation, or how it was applied in the study in which it was presented. Regarding the types of instruments, they were divided into three categories: ques tionnaires/scales, software/equipment, and twodimensional diagrams/area graphs. The type of approach of the instru ments can be quantitative, qualitative, or qualiquantitative. The instruments' targetusers were classified into children, learners, and the category of players in general, which con sists of instruments that did not have a specific public and/or 2 celulamultimidia.ufc.br/catalogo-ux-jogos/ Access date: 12/08/2020  can be used with every type of user. We implemented all these data types as filters to enable finding instruments according to their goals, the types of in struments they intend to use, and the targetedusers' profile. The full version of the catalog is available in Portuguese 3 . 3 celulamultimidia.ufc.br/catalogo-ux-jogos/ Access date: 12/08/2020

Discussion
Based on the data gathered from the instruments, its analysis and on the theoretical background about evaluation of the experience in digital games, we highlighted and discussed about some questions in this context, which we present in this section.

Why so many scales?
According to the data collected and analyzed in this research, scales and questionnaires are the most recurrent types of cat aloged instruments. This type of evaluation instrument can either be robust (with results with a high level of validity) and have superficial quality, generating questionable data re garding its validation (Lazar et al., 2017). Thus, the evalua tion results would depend on the quality of the questionnaire, its construction and validation and the team's understanding of how to use it. The usage of these instruments is broadly disseminated since the initial development stages of HCI science (Ozok, 2009), due to their accessibility and cost, as they do not need special technological equipment to be used. The results pro vide access to individual user information based on personal factors such as satisfaction, opinions, and ideas concerning the experience around some system usage (Ozok, 2009) these being some basic concerns in studies of UX.
According to Carneiro et al. (2019), besides the applica tion of this instrument type being rather convenient, there is also a frequent adaptation of questionnaires in the context of evaluating games. However, these adaptations usually don't follow any guidelines nor guarantee the psychometric prop erties of the original instruments (Carneiro et al., 2019). Ac cording to the authors, the substantial variety of constructs or components within the Player Experience can aggravate the issues arising from these adaptations.
The scales and questionnaires cataloged in this study are aimed to evaluate 63 different components of Player Expe rience and other perspectives of experience in games. The ease of creation (when informally done), adaptation, and use of this type of instrument may be one of the causes of this variety of components, which is further complicated by the lack of consensus on the constructs that constitute the PX and the different perspectives considered by authors.
Hence, if both Academy and Industry take more respon sibility towards creating and adapting these instruments, the psychometric measures are less jeopardized in the process. It is important to follow strict methodologies to create, adapt and validate the instruments.
General UX evaluation scales should be avoided in games because games and virtual environments have crucial par ticularities when compared to other systems. Games require a considerable mental activity rate (i.e. cognition, emotion, and motivation (Komulainen et al., 2008)), stimulated by re curring elements in the game context among (Takatalo et al., 2010). Attributes such as surprise, stress, and fear levels, may be desirable, which usually is not the case in other systems. Besides, attributes like these are probably not satisfyingly ex plored by scales and questionnaires only, requiring combina tion with other types of instruments, such as posttest images (Desmet, 2003) and specialized software (Ayzenberg et al., 2012).
Despite the prevalence of scales and questionnaires, these other types of instruments have been developed more con stantly throughout the years, so that this prevalence tends to decay. Whilst more types of instruments are developed, the amount of PX components evaluated increases, which may be due to the evolution of the technology applied in these in struments' development and how they can assess more types of data than scales and questionnaires. The evolution of the games throughout the years can be another reason for this increase, as well as the growth of the discussions in the lit erature about the experience in games and what composes it.

What am I evaluating when I evaluate PX?
The academic divergence regarding a concept that addresses experience in games and what it comprises is obvious. It is reflected in the variety of terms used to study it Player Expe rience, Gaming Experience, Game Experience, and User Ex perience. The literature states that UX in the game context, supported by digital technology, is responsible for provide the Player Experience and its multiple potentialities Drachen, 2011; Bernhaupt, 2015). The instruments cataloged in this study presented seven different terms to refer to the experience in games (Player Experience Game Experience, Gaming Experience, User Ex perience, User's Gameful Experience, Gameplaying Expe rience, and Gameful Experience), so that the most recurrent terms were Player Experience and Game Experience. Several papers introduced instruments that did not make it clear to which type of experience they referred. Often, components are described by very different sets of dimensions with no reasoning about the theoretical frameworks and experience perspectives being considered.
Although many authors have been working on formaliz ing the terms and the scope of Player Experience (Bernhaupt, 2015; Lazzaro, 2008b; Nacke and Drachen, 2011, this may still be one of the causes of the wide variety of components of the experience identified. Among the 70 different PX com ponents found, only 22 appear more than once in the in struments. This variety is even more evident in scales and questionnaires, which evaluate 63 of these PX components through 48 instruments. These results in several different assumptions behind the measurement of a PX component and reinforce major con ceptual divergences about experience in games. For exam ple, the instrument MEEGA+ considers that Player Experi ence can be evaluated by measuring Focused attention, Fun, Challenge, Social interaction, Trust, Relevance, Satisfaction, Perceived Learning, and User error protection . However, the Player Experience Inventory (Abeele et al., 2020) measures PX with a completely different set of components: Immersion, Meaning, Mastery, Curiosity, and Autonomy. Yet the instrument Video Game Uses and Gratifi cations Instrument (Sherry et al., 2006) proposed that Player Experience can be measured by Competition, Challenge, So cial Interaction, Diversion, Fantasy, and Arousal. Because of this lack of consensus concerning the definition and scope of Player Experience, it can be hard to know what is being as sessed when an instrument claims to evaluate PX and most of its components.
This fact indicates substantial differences between the psy chometric properties of a construct and raises questions about how trustworthy are the different instruments.
It is important to both practitioners and researchers be care ful to always select valid and widely tested instruments to evaluate experience in games. As researchers, we must be even more careful when creating and adapting PX instru ments and consider whether it is really necessary to create new measurement scales for widely addressed PX compo nents such as Immersion. Wouldn't instruments for evaluat ing experience in games be more robust if we focused our efforts on validating, translating, expanding, and improving already existing scales?
By creating more and more instruments instead of look ing for upgrades and diffusion of the existing ones, we may compromise the scientific progress of the field, as well as the usage of validated instruments by the industry .

How are cultural aspects being consid ered?
Once the culture is one of the main aspects of user context and deeply influences humancomputer interaction (Walsh et al., 2010), it is necessary to pay attention to one of its fundamental components: the language. Among the 58 cat aloged instruments, only eight were developed in a language other than English (Portuguese and Dutch) (e.g. Savi et al. (2011) or had a valid translated ver sion. Meanwhile, one instrument  is non verbal and is not confined to a specific language or requires translation. This large predominance of English instruments can be seen as an obstacle to the understanding of evaluated PX components and dimensions by untranslated instruments since language is a cultural expression, and it is essential to assimilate and diffuse the promoting experience (Coelho and de Mesquita, 2013). The discussion brought by Walsh et al. (2013) about the consequences of UX evaluations with people whose mother tongue did not correspond to the instrument language also applies to the context of experience in games. A significant increase in a player's cognitive effort is necessary to answer an untranslated questionnaire identified as the most used type of instrument in this study. The recurrence of this effort can be deduced to other evaluation technologies in which the user needs to translate (Walsh et al., 2013). When instruments are only available in English, they are only useful for people fluent in English. Even in this case, cultural differences be tween them and native English speakers can affect the valid ity of standardized questionnaires (Van de Vijver and Leung, 2001; Finstad, 2006.
However, just freely translating the instruments to players' language is also not a good alternative because the original psychometric properties of the instruments are not guaran teed, resulting in an invalid evaluation and making the data analysis untrustworthy (Walsh et al., 2013; Van de Vijver and Leung, 2001; Finstad, 2006. Hence, the wide range of PX components evaluated by the fifty scales and questionnaires in English may not be totally reliable if used with users who have a mother language other than English. In addition to that, the difficulty of evaluating Player Ex perience in different users' contexts may be one of the causes that we have identified only two instruments that are intended for children (Moser et al., 2012; Vissers et al., 2013 in the present study. Although PadillaZea et al. (2013) consider that questionnaires enable access to qualitative data such as the users' satisfaction aspects and emotional impact in a pos terior discussion with each participant, it is hard to analyze the collected data when it comes to children. This difficult occurs because children may not be reliable when answering questions (PadillaZea et al., 2013). When applying an eval uation instrument, the children's behavioral aspects must be considered. As Barendregt (2006) states, they have a more reactive and impulsive approach than a logical one, so they usually have problems at verbalizing their thoughts while in teracting with digital technology (Barendregt, 2006).
There is room for the community to develop player experi ence evaluation instruments that consider the particularities that portray children and other types of users with behav ior peculiarities as well as to develop research to validate translations of valid English instruments.

Conclusion
This study presents an analysis of the data gathered from a set of 58 instruments to evaluate the experience in digital games, in addition to discussing about some questions regard ing the terms used do describe the Player Experience, its com ponents and dimensions, about the application of the instru ments in an evaluation process and the impact of cultural and contextual aspects on the evaluation. We also developed an extended version of the catalog of Player Experience evalua tion instruments developed by Borges et al. (2019), improv ing its navigation, adding 13 new instruments and displaying more detailed information about each instrument.
The analysis of the instruments data raised discussions that can be relevant for Game User Research and Player Experi ence future studies and related studies in User Experience and its concepts, evaluations, market, and academic trends. We expect that the discussed ideas presented in this article may support and enhance other discussions about the scope and definition of Player Experience and its components or in volved or corelated areas. The results of this research can be useful for some discussions about the translation and adapta tion of instruments to other sociocultural contexts or specific publics, the development and adaptation of scales and ques tionnaires for different research goals, and also about the val idation of instruments.
This study aims to support researchers and professionals in making informed decisions when choosing PX evaluation instruments in games and virtual environments with the dis cussions, data analysis, and the catalog of instruments pre sented here. For our future work, we plan to expand the cat alog, including new instruments, extract and analyze addi tional data of the instruments, outline correlations between the terms used to describe Player Experience and its com ponents and also draw comparisons between instruments for different applications.