Building information visualization of elearning data with Vis2Learning guidelines

Information Visualization provides techniques to make better charts that enhance human perception about patterns in data and consequently support the user interpretation. In the educational area, visualizations help pro­ fessionals to analyze a great amount of data to inform decisions to improve the learning­teaching process. The literature has shown that there is a gap in the development of educational data visualizations that fulfill end­user needs. This paper presents Vis2Learning: a scenario­based set of guidelines for the development of visualizations in the e­learning context. Vis2Learning provides a set of scenarios from which educational data visualizations can be developed, for each scenario, we provide the recommended chart, its aim, characteristics and examples of its application in the e­learning context. Besides, we provide a set of guidelines to improve users’ interaction with each chart. We applied an online questionnaire with 34 end­users (Brazilian teachers), evaluating visualizations that were created by using the Vis2Learning. The results reveal: (1) the visualizations, based on Vis2Learning, were more suit­ able to be applied in the e­learning context; (2) some non­traditional visualization formats are difficult to interpret by users who did not have previous experience with visualizations in the e­learning context; and (3) experience in teaching is not strictly related to knowledge of charts about educational data.


Introduction
Information Visualization (InfoVis) area assists designers and developers in creating charts based on the needs of a target audience (Ware, 2012; Munzner, 2014. Based on HumanComputer Interaction (HCI) techniques, InfoVis aim is to support the development of charts to provide users with better graphics representation to enhance human perception of large amounts of data (Ware, 2012; Strey et al., 2018. The visualization of data in flat tables does not help the user to infer patterns or find outliers by observing the data (Card et al., 1999). One of Infovis premisses is the use of graphical representations to explore the abstraction promoted by the charts, which reduces the effort to obtain information based on the data (Carneiro and Mendonça, 2013).
The literature shows a growing interest in supporting the Education community with tools based on InfoVis, to allow the analysis of "educational data" and consequently to as sist decisionmaking regarding the teachinglearning process (Reyes, 2015). According to the literature, educational data comprise information related to student's interactions with resources in the educational context, such as: activities; test score and demographic data (Jordão et al., 2014; Vieira et al., 2018; Tervakari et al., 2014. Schwendimann et al. (2017) point to a gap in studies that explore the HCI in the context of visualizations about educa tional data. The authors point out that one of the challenges faced is the lack of investigations on techniques that help the development of graphic visualizations in order to fulfill the users' needs related to elearning environments. Elearning environments are computer systems that can generate and store large amounts of data related to students' interaction with educational resources (Vieira et al., 2018).
We published a previous work entitled "Vis2Learning: A scenariobased guide of recommendations for building edu cational data visualizations" (Macedo et al., 2020) where we discuss details about the creation process, validation stages and evaluation of Vis2Learning by the lens of the visualiza tion users. The Vis2Learning has 15 scenarios where, for each scenario, we provided: the recommended chart and its characteristics; application examples; and guidelines to en hance the user interaction with each chart. In this article, we present an extended version of Macedo et al. (2020). The novel contributions in this article are: (1) an expanded ver sion of the analysis of the participants' profile; (2) a new correlation analysis between the participants' experience and the evaluation results; and (3) an expanded discussion about cases where the participants declared a low level of agree ment about the charts created based on Vis2Learning.
The method used for creating Vis2Learning considered three stages. A first version was created (i) from a literature review, regarding visualizations created for elearning sys tems. The first version was validated (ii) by three experts in the fields of Educational Data Mining (EDM), HCI and Info Vis, to generate a refined version. Two sets of charts were cre ated, one of them based on the final version of Vis2Learning and the other did not apply the guidelines. These sets of charts were evaluated (iii) through a questionnaire that col lected the perceptions of 34 endusers (e.g. teachers from Brazilian schools) in relation to the interpretation of the data presented by the charts.
The results of the evaluation suggest: (1) traditional vi sualizations, based on the guidelines, were perceived as the most appropriate for elearning environments; and (2) more teaching experience is not correlated to knowledge about us ing visualizations. The contributions of this article are: (1) to describe the systematic method applied in the creation of Vis2Learning; (2) to obtain and gather information, regard ing the development of visualizations about educational data, which are the result of experiments with endusers; (3) to pro vide a scenariobased approach, which aids the developers (via guidelines) in understanding better ways to create a good information visualization for the elearning context; and (4) to present the users' feedback about the visualizations cre ated applying the Vis2Learning.

Background
In this section, we provide concepts about Information Visu alization and present some related work.

Information Visualization Concepts
The InfoVis area aims to develop methods and techniques to enhance the interpretation of data based on the human per ception. The main objective is to prevent users from employ ing excessive efforts on the stage of data preparation to ded icate more attention in the interpretation of data for decision making (Card et al., 1999). According to the InfoVis prin ciples the developed visualization should amplify and allow the perception of emerging properties of the data, such as patterns, deviations, groupings and trends (Ware, 2012; Card andJacko, 2012). Munzner (2014) defines a framework to raise awareness about the importance of contextualizing the "idiom of visu alization" to the user's data, tasks and domain during the de sign and evaluation of charts. Idiom of visualization can be perceived by visualization formats and features as zoom and pinch or data interaction features as navigation between dif ferent granularity of information.
A visual approach is recommended to prevent the user from using statistical techniques to analyze educational data, since these data are numerous and complex (Barbosa et al., 2017; Reyes, 2015. A poor definition of the visualization idiom can result in tools with low adherence to the target au dience as the tool may not be useful for the user's task (Mun zner, 2014). Tervakari et al. (2014) report that the visualiza tions that will be used by the target audience of elearning sys tems need to be simple, direct and show relevant data without the need for the user to have statistical knowledge.

Related work
We found different processes in the literature for creating vi sualizations about educational data. Using prototypes Alves et al. (2018a) and Alves et al. (2018b) developed a process to drive the development of visualizations based on the context of use. To develop views for MOOCs Chen et al. (2016) pro pose an iterative process in which visualization prototypes are successively refined through interaction with users. At each stage of discussion and refinement, prototypes are grad ually populated with data. With the same theme Ruipérez Valiente et al. (2017) presents a process that uses analysis of the materials available in the learning environment and inter views with users to choose the visualization formats. Maldon ado et al. (2015) and Conde et al. (2015) proposed a work flow for the elaboration of visualizations whose data came from learning analytics (LA) algorithms.
Considering the construction of visualizations, Klerkx et al. (2017) proposes a set of guidelines to assist the cre ation of specific visualizations aimed at analyzing the learn ing path. The authors define a set of steps to determine the moment when the developer needs to define the necessary data and choose the visualization formats to fulfill the con text needs.
Munzner's book (2014) has been a reference for the de velopment of research in the field of visualization. However, the guidelines proposed by the author are abstract, as they do not deal with visualization formats aimed at a specific context and target audience. In addition, the guidelines are distributed throughout the different chapters of his book, re quiring the complete reading of the work.
The related work above presents processes that apply HCI techniques to organize the creation of visualizations (Alves et al., 2018a,b; Chen et al., 2016; RuipérezValiente et al., 2017; Maldonado et al., 2015; Conde et al., 2015; Klerkx et al., 2017. However, none of them deals effectively with the visualization formats that can be applied within the con text of educational data in elearning environments. Mun zner (2014) presents aspects related to the idiom of the visu alization disconnected from any context. The Vis2Learning differs at this point, it offers guidance on which visualiza tion formats to use in each scenario of applicability in the e learning context. In addition, the Vis2Learning provides in formation on how to work on the idiom of visualization to improve interaction in each recommended visualization for mat.

Method
The method used for creating Vis2Learning was divided into three stages, as shown in Figure 1.
We searched for articles with discussions about educa tional data visualizations created for elearning systems. From the pertinent articles found, we extracted lessons learned, reported from the design and evaluation activities. The first version of Vis2Learning was created as a list of plain text containing hints to construct charts.
In the second stage, the first version of the Vis2Learning was validated individually by three experts in the fields of EDM, HCI and InfoVis. The experts gave a feedback based on their evaluation. This feedback was used to re fine the guidelines, resulting in another version. This refined version was, again, validated by three experts, generating the final version, which is structured as guidelines, named Vis2Learning.
In the third stage, we continued evaluating Vis2Learning. The objective was to check if applying Vis2Learning, for creating visualizations, results in visualizations that are cor rectly interpreted by the target audience. A set of visu alizations was created, where half of them applied the Vis2Learning guidelines, and the other half did not. The half that did not use the guidelines was created using the Google Sheets wizard for creating charts from a set of data. These

Creating Vis2Learning
In this section, we present the first two stages of the Vis2Learning creation (see Figure 1).

The first version
The investigation of the literature for the elaboration of the first version, followed the protocol of a Systematic Litera ture Review (SLR) (Petersen et al., 2008). It is important to highlight that we did not perform a SLR, but we followed the same rigor to construct the search string. The objective was to find articles that describe the process used to design and/or develop visualizations of educational data for elearning en vironments. The keywords of the search string were based on the previous reading of seminal articles with results that matched the creation of the guidelines.
After refining the search string (17 iterations, includ ing and removing keywords), the final string was: "("e learning") AND ("Information Visualization" OR "Data visualization" OR "InfoVis" OR "Visual Analytics" OR "Learning Analytics" OR "Academic Analytics")". The search string was used in 5 scientific digital libraries: ACM Digital Library; IEEE Explorer; Scopus; Science@Direct; and the publications portal of the Brazilian Special Group on Informatics in Education (CEIE). We found 1207 articles 1 .
We used these articles to create the guidelines, by follow ing three steps: extraction; organization; and consolida tion.
The extraction step was carried out by one of the re searchers, from February to May 2019. First, the researcher read the title, abstract and results/conclusion of the 1207 ar ticles, to search for lessons learned related to design and/or evaluation of visualizations about educational data. When an article reported lessons learned, it was tagged to be fully read in the next step. The tagged articles were fully read and all excerpts found were stored in a spreadsheet. To allow the tracking of the excerpts, during the validation step, each ex cerpt received an identifier composed of "[ArticleNumber ExcerptNumber]". For example, the first excerpt of the first article received the identifier [11]. In addition to the identi fier, we included: the digital library name where the article was found, the page number of the article were the excerpt was found, the category in which the excerpt was inserted, the visualization format that the excerpt was related to, the excerpt taken from the article and a compilation of ideas elab orated by the authors based on the full article context.
We assigned categories to the excerpts with the aim of distinguishing the lessons learned types. The first category, called databased, was assigned to excerpts that were related to some characteristic derived from the semantics of the data and were inherent to visualization. The perceptionbased category was used when the excerpt was related to the user's interpretation of the visualization format. Table 1 presents an example of an excerpt. Science@Direct 6 perceptionbased area chart Article Improving the expressiveness of blackbox models for predicting student's per formance Article excerpt Progression charts can represent individual or group results. In the case of indi vidual charts, it allows the detection of risk of failure of a student, and the effect of guiding. For instance, in Fig. 3, the student is classified as low or high perfor mance in the first five weeks, with a low or medium level of confidence. However, the prediction of belonging to high performance class is dramatically reinforced from week 6, when the student reacts, so the performance results become much better in the last weeks. Another interesting use of progression chars is for study ing group results (Fig. 4). In this case, the chart represents the average probability for every class and every week, considering all the students in a given group. The interpretation is similar to that of the individual charts, but in this case trends about the whole group can be detected. Besides the comparison between groups, it is very interesting comparing the progression of the whole group in accordance to the learning plan.

Recompilation of ideas
The area chart (called "progression" in excerpt) is ideal to represent the student's evolution over time. It can be used individually to provide feedback about stu dents' pace and performance, allowing the student/teacher to realize the need to change the pace of studies. Also, it can be used to visualize how groups are per forming against expectations and find patterns among everyone.
After performing the extraction step, the organization step was conducted in order to group the findings consider ing the visualization format and the category assigned to the excerpt. At this stage the excerpts were organized without removing or adding any information. This step also aimed to determine if there were similar or even duplicate excerpts, and then form a list of unique excerpts.
In the consolidation step, all excerpts referring to the same visualization format were recompiled and unified to gener ate the first version of the guideline. Each lesson learned was rewritten as a guideline, containing: identification code; chart name; the codes of the excerpts that were grouped; and a paragraph describing the guidelines. The guidelines were identified as guidelines Gn (17 in total), and general guide lines 2 GGn (9 in total). Table 2 presents an example of a guideline that was consolidated, in which the excerpts [15], [17] and [122] were brought together to generate guideline G6. Table 3 summarizes all the guidelines (G) and the general guidelines (GG), containing the visualization formats and the related excerpts. The area chart is ideal for representing intermediate results and student progress, as it can represent the magnitude of change over time. Ideally the X axis should be the time, while the Y axis can state many variables representing mea surements, for example the distribution of time spent on each question of an activity. This chart helps the student to be aware of their progress over time. When used individu ally to provide feedback, it allows the student/teacher to re alize the need to change studies' approaches. It can also be used to visualize the performance of groups in relation to ex pectations and find patterns, which can represent difficulties of groups of students. When representing many variables, different colors and transparency can be useful to avoid los ing information due to overlay. [63] GG7 [73] GG8 [72] GG9 [22]

First version Validation
The first version was validated by three experts in the fields of EDM, HCI and InfoVis. The three experts are senior re searchers and will be named according to their field of exper tise: EDM EDM researcher and expert in building systems for recommending educational content based on data anal ysis; HCI HCI researcher specialized in UX and informa tion technology in education; and InfoVis HCI researcher specialized in InfoVis, educational data and building systems for LA. The researchers were selected by specialty and con venience. The validation process was divided into three steps as described in the next paragraphs in this subsection. In the first step, an electronic spreadsheet was prepared containing the first version of the guidelines and, for each guideline, the following were described: (i) the identification code; (ii) the name of the chart; (iii) an example image (which could be accessed by clicking on the name of the chart); (iv) the description of the guideline; (v) a column where the ex pert should deliberate his opinion (Approved, Partially ap proved or Disapproved); (vi) a column for comments on the 2 Identification attributed to guidelines that could be applied in more than one visualization format covered by the first version of the guideline. content of the guideline (semantic); and (vii) a column for comments on problems related to the writing of the guide line (sintax).
The experts worked on their spreadsheet without access ing the spreadsheet of the others. In addition to filling out all columns with the feedback on each guideline (items from v to vii), all experts have also created generic fields to input com ments on the structure and organization of the guidelines.
The second step was conducted after the experts' first re view. The experts' generated a list, containing 71 related comments on the adjustments that should be applied to the guidelines. All notes made by the experts were complemen tary, with no conflicting or even redundant comments. The validation scores are presented in Table 4. All experts sug gested that the format of presentation of the guidelines (in paragraphs) was suppressing important details of the ex cerpts. The format was changed in the final version.
The third step started with the definition of a new format for presenting the guidelines. In addition to the new presen tation format, seven of the guidelines (G and GG) were ex cluded for presenting illdefined contributions or for not mak ing it clear the applicability for the area of visualization about educational data in elearning environments (i.e. G3, G7, G8, GG2, GG5, GG7, GG9). The general guidelines received the questioning in relation to which visualization formats they could be applied to, so it was decided to remove the "Gen eral guidelines" classification and contextualize the content of these guidelines for each proposed visualization format. At the end of the third step the guidelines were transformed into a set of guidelines based on scenarios of applicability of visualizations about educational data in elearning environ ments. The guideline was validated again by the experts and approved without further adjustments, generating the final version of the proposal called Vis2Learning.

Vis2Learning guidelines
We defined a template to the writing of the guidelines final version. The template did not impose any limitation on the amount of information described in it. This avoided specific details, taken from the literature, to be out of our guideline specification. When a visualization does not have guidelines for a particular element, it is kept blank. Elements that should not be used, as they do not represent good practice, are filled with the label "normally not applied". The fact that an ele ment is blank does not mean that there are no guidelines, but that during the search there were no experiences that could contribute to that approach.
Vis2Learning provides guidelines to support developers in the creation of visualizations that will be used by teach ers, which means that teachers are the target audience. Dur ing the literature search, we run a string that did not include any keyword regarding "teachers". However, the literature review showed that the lessons learned, found in the papers, put teachers as the main endusers of the visualizations.
The final version of Vis2Learning contains 15 scenarios of application for visualizations about educational data in the context of elearning. Each scenario has an identifier (SC) and is composed of the following elements: chart name; representation purpose; chart characteristics; application ex ample; and guidelines. This last element has four subtopics related to the idiom of the proposed visualization for that scenario, which are: exact value; filtering data mechanisms; highlight data; and some particularities of that scenario.
In the first version of the guide, there were general guide lines that dealt with different aspects of the visualizations (colors, shapes and/or sizes) and were not specific to just one type of visualization. However, due to the relevance of these guidelines, they were incorporated for each visualization for mat and stored in the subtopics of the element called "guide lines".
The Vis2Learning has a total of 59 guidelines distributed among 15 scenarios of applicability for different formats of visualizations. For each visualization format there is a char acterization field that was extracted from the literature on the types of data that it can represent. In addition, the guideline describes 24 examples for application of formats within e learning systems. Table 5 presents scenario SC6. The full ver sion of Vis2Learning is available at the link 3 . Table 6 presents a summary of the application scenarios in relation to the vi sualization formats contemplated.

Evaluation with endusers
This evaluation proposal was to collect the participants' perception about the charts that were created using Vis2Learning. Data was collected using an online question naire (Lazar et al., 2017). This evaluation followed the ap proval procedures of the Human Research Ethics Commit tee of UFSCar and was authorized by protocol number 31252720.1.0000.5504.  Illustrates the midway students' outcomes, showing the evolving and data changes over time Aim represents variables, associating them to progress viewing (e.g. percentage completed, time elapsed) Chart characteristics a continuous line passes through all the crossing points presented between the X and Yaxis it is recommended that the Xaxis contains the progress scale (e.g. time, steps.), and that the Yaxis contains the quantitative variable the use of several lines is suitable to describe different groups of data Example of application provide students with a visualization of the distribution of time that they spent on each part/step of an activity from this visualization the students are able to evaluate their study pace and consequently make adjustments if they need it is also possible to visualize the performance of workgroups versus an expected outcome to search for difficulties that they could have in one activity Guidelines Exact value display the real values on tooltips when users hover the mouse on the chart Filtering data mechanisms provide ways to see the data of particular students

Highlight data
in cases of multiple variables representation, colors with transparency are recommended to avoid missing information caused by the overlapping of areas, for instance Specificities use patterns (e.g. dots, dashes) to contrast the areas in cases colors are not available; the 3D bar format is not recommended because it can insert difficulties to the comparison of data; provide the meaning of the colors and the format of the lines in captions

Planning
For evaluation, we created 15 examples of use (EU) of charts: one EU for each scenario (SC) in the guidelines. To ques tion the participant about the interpretation of the information contained in the chart, for each EU there was an assertion. In order to carry out a comparative assessment, for each EU, two distinct charts were created. One of the charts was built following the guidelines of Vis2Learning, henceforth called Vis2LChart. The other chart was created using the Google Sheets wizard that generates a visualization from a data set, henceforth called GSheetChart. We decided to create these two charts to analyze if the target audience will have a differ ent interpretation of the charts created using Vis2Learning and the other created using a generic approach that does not consider the context of use. All these elements (EU, charts and statement) were vali dated by the three experts who validated the Vis2Learning (see Section 4.2). Figure 2 presents an example of all ele ments created regarding EU4. First, each expert, individu ally, analyzed and commented on all the elements created for the evaluation. These comments were compiled in a report. This report was presented to the three experts in an online meeting, where they had to accept the EU and their respec tive elements or suggest changes.
After performing the refinement of the elements created for the evaluation, two questionnaires were developed. The two questionnaires (A and B) contained the same EU and the same statements. However, the charts were distributed between the two questionnaires (A and B) alternately, so that both had Vis2LCharts and the GSheetCharts at random. For example, for questionnaire A and EU1 a GSheetChart was presented, and for the EU1 in questionnaire B the Vis2LChart was presented.
In the questionnaire, the EU, and its respective chart, were presented to the participants who should select their level of acceptance in accordance to the statement related to them. The degree of acceptance could be indicated using a 4point Likert scale: I totally disagree; I partially disagree; I partially agree; and I totally agree. Johns (2005) and Garland (1991) point out that a small scale without a central point (neutral, I neither disagree nor agree or I don't know) takes the advan tage of getting a more precise response from the participants. Johns (2005) argues that the neutral point is commonly used by the participants to avoid a possible conflict of opinion with the researcher. Thus, in our questionnaire, the 4point scale was chosen.
The questionnaires A and B had 4 sections: (1) a welcome message and a brief explanation of the research aim; (2) the Informed Consent Form (ICF) explaining the research condi tions and gathering the participant's acceptance (if the partici pant did not accept the ICF, the questionnaire was ended with a thank you message); (3) a demographic gathering about the participant's profile and their level of academic and profes sional experience; and finally, (4) the 15 EU to collect the par ticipants' perception about data presented by visualizations.

Execution and Analysis
The participants were invited via email. To control access to the questionnaire, we designed an algorithm to calculate, in an ordinal way, the number of access and redirect the partici pant to questionnaire A if it was a even access number and if not, redirect the participant to questionnaire B. We adopted this strategy to balance the number of responses in each ques tionnaire and to prevent a participant from answering both questionnaires. We registered 123 accesses during the period the questionnaire was open. There were 34 questionnaire an swers, being 17 for questionnaire A and 17 for questionnaire B.
We analyzed all the data collected, i.e. all the responses from questionnaire A and B. By joining all the data, we could have an overview of the evaluation 4 . Descriptive and inferen tial statistics Lazar et al. (2017) were used to explore the data from different lenses. The boxplot was used to observe medi ans and outliers of the participants' perceptions. To analyze whether the acceptance rates on the visualizations were influ enced by the profile of the participants, the Fisher (1922) ex act test was adopted. The details of the results are described in the following section.

Results
The results obtained through the analysis are presented in the next subsections in three perspectives: profile of the partici pants, perception of the participants about the charts and in fluence of the experience of the participants on their percep tion about the charts.

Participants' profile
Regarding the 34 participants, 44% were above 41 years old and the most common level of education was postgraduate with about 30%, followed by the master's degree with ap proximately 24%. The levels of education in which the par ticipants work are varied, however, higher education (38.2%) and middle school (20.6%) stand out as the main activities. Four participants stated that they were not acting as teachers, however, this answer reflects only the current moment of the participant since when analyzing the experience time teach ing it is noted that the majority of participants (58.8%) had more than 6 years of experience. Only one participant, who was a student of pedagogy, had no professional experience in teaching.
All information collected from participants is presented in Table 7, each participant is identified with an ID (Pn). The table comprises information about: Identification ID, age and the level of instruction; Job context the graduate level that the participant work and time of experience; Knowl edge The level of knowledge informed by the participant about software development, InfoVis area, interactive charts and educational systems with data visualization about educa tional data; Use of charts the participant preferences about use of charts in their job and personal use.
Participants were asked about experience on topics related to visualizations. Most of the participants had theoretical and/or practical knowledge about computer programming and the creation of interactive charts (See Figure 3 A   An open and nonmandatory question asked which sys tems the participants used to generate graphs. Each partici pant could list more than one system or none. There were 32 answers, and the most frequent answer was the use of spread sheets (16 citations) and only 3 participants mentioned sys tems for educational context with the use of charts. Figure 4 presents an overview of the participants' perception about the visualizations related to the Vis2LCharts and the GSheetCharts. The participants expressed agreement in re lation to the statements associated with the Vis2LChart and disagreed when the associated chart was GSheetChart. How ever, the median of the two groups pointed to the answer "I partially agree". The median found suggests that the partici pants showed a tendency to consider the interpretation using the GSheetCharts as partially adequate. A panel of boxplots, containing one chart for each sce nario, was created to examine in depth the data that make up the median ( Figure 5). In order to create Figure 5, data from the participants who interacted with GSheetCharts were nor malized through the inversion of their semantic value. That is, the answers "I disagree" were replaced by "I agree" and so on. After the normalization, we aggregated the responses of all participants. Normalizing the data, we avoid the boxplots extend across the entire range of responses, which avoids hid ing the outliers. The boxplots located between the middle and the top in Figure 5, as in the case of EU2, EU3, EU6, EU7, EU8, EU10, EU12, EU13, EU14 and EU15, represent scenarios where the Vis2LCharts was considered suitable for the context of elearning, while the GSheetCharts received a lower agreement level. Table 7. Participants profile | * ESwDT: educational systems with data visualization about educational data | Knowledge: (1) I've never heard of; (2) I have theoretical knowledge; (3) I have practical and theoretical knowledge; (4) I have deep knowledge | Use of charts: (1) I don't use it; (2) I used it a few times; (3) I always use it because it is necessary; (4) I always use

Influence of the participants' experience on the perception of the visualizations
The data collected in the evaluation were also analyzed by crossing the profile of the participants' perceptions about the Vis2LCharts. We verified if the participants' previous ex perience with visualizations systems about educational data could influence the choices they made for each EU. To con duct this verification, the Fisher (1922) exact test was per formed. We chose this test because it was suitable to compare categorical data, collected from small samples (i.e. <1000).
Besides that, it calculates the exact significance of the devia tion from a null hypothesis using pvalue, while other meth ods use an approximation. In addition to providing greater accuracy in small samples, the exact tests do not require a balanced or welldistributed sample (Mehta and Patel, 1996). Because it is a small sample (34 participants), a 95% (0.05) confidence interval was adopted to mitigate errors in the re sults 5 . The hypotheses formulated were: • H0 The previous experience of the participant on visualizations in ed ucational systems has no influence in the acceptance of Vis2LCharts; • H1 The previous experience of the participant on visualizations in educational systems has influence in the acceptance of Vis2LCharts. Table 8 shows the data used for this test. The pvalue was 0.0392, below the confidence interval (0.03 < 0.05). Consid ering this result, there is enough statistical significance to re ject the null hypothesis. Thus, we can infer that the experi ence of the participants with educational systems with data 5 We run tests from this website https://astatsa.com/ FisherTest/.
visualization influenced the positive perception in relation to the Vis2LCharts. We also decided to check whether the participants' ex perience in teaching has influence in the acceptance of Vis2LCharts or not. In the participant's profile section of the questionnaire, the highest possible option for the partic ipants' experience is "more than 6 years", for this analysis, we considered the participants that chose this option. The hy potheses formulated were: • H0 The large experience in teaching (i.e. more than 6 years ) has no influence in the acceptance of Vis2LCharts; • H1 The large experience in teaching (i.e. more than 6 years ) has influence in the acceptance of Vis2LCharts. Table 9 shows the data used for this test. The pvalue was 0.3270, above the confidence interval. We can conclude that there is not enough statistical significance to reject the null hypothesis. Thus, we can observe that more experience in teaching does not, necessarily, ensure knowledge about the use of visualizations to gather information. This result rein forces the need to make the users aware about the possible uses and aims of each visualization format.

Discussion
Related work (see Section 2.2) shows that most of the visu alization proposals are focused on processes to organize the development of visualizations for elearning systems (Alves et al., 2018a,b; Chen et al., 2016; RuipérezValiente et al., 2017; Maldonado et al., 2015; Conde et al., 2015; Klerkx et al., 2017. Even though it is in the context of elearning, none of these works provided guidelines of suitable formats of visualization for the elearning field.
In the related work, the authors suggest that the develop ers search the literature for appropriate chart formats (Mal donado et al., 2015; Conde et al., 2015; Klerkx et al., 2017; RuipérezValiente et al., 2017. Vis2Learning proposes a pragmatic way to assist developers in choosing the visualiza tion format for elearning systems, based on data and guided by application scenarios. None of the related work applied a systematic way to explore and organize the lessons learned about the use of visualizations, regarding educational data to support future implementations and that makes Vis2Learning different from these works.
Considering the formats used by GSheetCharts, we see the use of pie chart in EU1 and bar chart in EU11. Vieira et al. (2018) states that pie and bar charts are considered tra ditional and well accepted by teachers in educational context. In Paiva et al. (2019)'s survey, teachers pointed out that vi sualizations with nontraditional charts are useful but they stated they were more confident when using a traditional chart.
The visualization formats proposed for Vis2LCharts that received the lowest agreement level were: violin plot (EU 4); radar chart (EU5); and activity diagram (EU7). All the visualization formats mentioned are considered as non traditional and their use, in the educational data visualization context, is relatively new (Vieira et al., 2018; Dourado et al., 2018. In EU4 case (See Figure 2) we see a nontraditional for mat used as Vis2LChart, called violin plot. As recommended by Vis2Learning and discussed by Barros et al. (2017), the vi olin plot is a suitable format for visualizing the distribution of data that have more than one group in its composition (e.g. students who passed and failed). This format is suitable for this EU, because it describes a case of a teacher that needs to visualize the distribution of students' grades in relation to passing scores. Despite that, participants show a low level of agreement for Vis2LChart and this finding confirms the results reported by Paiva et al. (2019).
In EU5 case the participants declared a low level of agree ment for both visualizations. The GSheetChart, which was a line chart, received more positive answers in relation to the Vis2LChart which was a radar chart. The low level of agree ment related to the Vis2LChart may have happened due to the lack of familiarity, from the audience, of this chart for mat. Dourado et al. (2018) stated that the radar chart is not among the most used in educational data visualization sys tems, while the line chart is widely used.
In EU9 case (see Figure 6) the participants show a posi tive level of agreement for both charts presented. Vieira et al. (2018) conclude that the dot chart presented as GSheetChart (see Figure 6 B) is one of the three most traditional visualiza tions used in systems with visualization about educational data. Similar to the EU5 case, familiarity with the chart pre sented as a GSheetChart may have affected the participants' level of agreement.
The majority of the participants declared to have more than 6 years of experience in teaching and our analysis showed that this experience is not directly related to knowledge about the use of visualizations to explore educational data. We pointed out that this experience does not contribute to over come the cold start barrier about nontraditional charts.
Our analysis showed that the experience of the partici pants with educational systems that contain data visualiza tions, positively influenced the participants' perceptions, re garding Vis2LCharts. Vis2LCharts were created based on Vis2Learning and includes traditional and nontraditional vi sualization formats.
We conclude that when proposing a visualization, in addi tion to ensuring that it is appropriate to the user's context, it is important to provide the user with some information about the visualization format. A solution would be to provide a brief explanation and examples of what is the aim of the vi sualization format used. Munzner (2014) is the most wellknown reference on guide lines for information visualizations. However, the author's work provides the guidelines without setting them into a field or context. Being our proposal focused on the context of use, we decided to compare the Vis2Learning with the guidelines from Munzner's book.

Comparison with Munzner
To conduct the comparison, we followed some steps. First, we read the chapters of the Munzner's book and extracted the "rules of thumb" 6 . Munzner's rules of thumb were compared with the content presented in Vis2Learning by searching the linking between the guidelines. We consider that the two pro posals have a linking when the Vis2Learning guidelines en compassed at least one of the rules of thumb.
We also created a categorization to classify the linking strength between the guidelines of both proposals. A cate gory was assigned to the relationship when a guideline from Vis2Learning: SP had the same purpose as a rule of thumb; EC embedding concepts of a rule of thumb for visualiza tions in elearning systems; or SA presented some aspects of a rule of thumb. The rules of thumb and the data generated by the comparison process are available at link 7 .
In Figure 7, we observe that all the Vis2Learning scenar ios (X axis) established at least one connection (represented by bubbles) with Munzner's rules of thumb (Y axis). Note that 18 of the 35 connections presented the category EC (red bubble), indicating that the rules of thumb are embedded by Vis2Learning.
An example of this is the SC9 that brings information about the map chart, it incorporates: a rule of thumb T6 with guidelines on highlighting a location selected by the user through interaction; T14 with suggested functionalities of zoom to guide the user observing the relevant data according to a filter; and T15 with the guideline of applying tones of the same color to maintain the quantitative semantics of the visu 7 https://docs.google.com/spreadsheets/d/1heg3Hc-w0ZA0VOpNsi43A4kf9We7GaGG1IyYhzVXBu8 alization through contrast in scenarios that do not favor the differentiation of colors. The SC2, SC3, SC6 and SC9 sce narios stand out for each embedding three or more aspects of the rules of thumb.
The focus of Munzner's work (2014) is on the user's in teractions with the visualizations, however, its content is ab stract and has no guidelines on the use of visualization for mats in specific contexts. The connections between the rules of thumb and the Vis2Learning suggest that the guidelines presented in this article covers the work of Munzner (2014), with the difference of having information on the applicability of contextualized visualizations for elearning systems.

Conclusion
This article presented Vis2Learning: a scenariobased set of guidelines for applying visualizations about educational data in the context of elearning. Its differential is to recommend visualizations based on applicability scenarios. The guide lines and details that make up the guidelines were developed through a literature review, validated by experts. The compar ison of Vis2Learning with Munzner's rules of thumb demon strated that the guidelines are in line with what the author recommends about having a focus on user interactions for the context of data visualization.
We noted, during the evaluation, that teachers tended to disagree about the application of nontraditional visualiza tions, that is, new visualizations that are normally used (or even known) by the users. However, according to the litera ture these were the most appropriate visualizations to be ap plied to the scenarios developed to the evaluation. Our anal ysis showed that more experience in teaching (more than 6 years, according to our analysis) does not necessarily influ ence the teachers' knowledge about the use of visualizations to gather information. However, the experience on visualiza tions in education systems promoted a high level of agree ment of Vis2LCharts. With this, we observed that informing the users about the charts' characteristics is as important as ensuring that the visualizations are appropriate for the con text.
As future work, we intend to create an online repository with information about the use of the visualization formats covered by the guide, providing endusers with a help link about each chart. To mitigate problems related to the use of visualization formats that are little known by the enduser, we intend to expand the guidelines, considering the preferences of user groups. Also, we intend to expand the investigation with endusers to generate artifacts that allow transforming the guidelines into a catalog of design patterns.