An Evaluation Study of a Electronic Whiteboard System for Nursing Care in the Emergence Unit

. The main objective of this research is to evaluate a task and alert management system denominated SHAVI (SHAred VIew), developed to be used by the Nursing team of the Emergency Unit of a University Hospital located in southern Brazil. The system was developed as a resource for managing patients clinical data and warning signs of clinical deterioration during urgency and emergency care. The research is characterized as exploratory and descriptive, with data collection carried out in the field with 21 participants, including students and professionals working in the emergency sector, through the application of a questionnaire consisting of 48 questions. The usability and user acceptance of the tool was mainly verified based on Nielsen’s Usability Heuristics and the Technology Acceptance Model. The evaluation and user participation contributes to identify problems and make corrections before putting the system into use, contributing to its adoption and acceptance.The evaluation reveled that the system has great potential for use and can be considered a safety mechanism for monitoring patients in the emergence unit, but to be adopted, needs to be integrated with other systems used in the Hospital and to train first-time users.


Introduction
The adoption of Information and Communication Technologies (ICT) as a tool to improve the work routine of professionals is a challenge for health organizations, considering that they are extremely complex [Barlow 2013]. Specifically in hospital emergency units, patients can be exposed to different risks and sometimes suffer adverse events [France et al. 2005]. This scenario makes clinical or care decision-making complex, as it requires systemic knowledge about the attributions developed by the team, in addition 24:2 to their specific skills for patient care, which implies a large volume of information to be managed.
Studies described in [Leape 1994] show that human failures or errors occur in a considerable percentage of care provided and can cause harm to patients' health. Additionally, it is known that health professionals, especially those who work in an emergency environment, have a high cognitive demand and most tasks performed require a quick response time. In this context, the introduction of Health Information Systems (HIS) should promote improvements in the development of activities and information sharing of these professionals. The information shared between the health team about the treatments and the patient's trajectory are important sources of knowledge, often lost when they are erased from traditional blackboards or because they are filed on paper and updated by different people, in condensed formats and without standardization. Otherwise, when managed by a computerized system, they provide the development of a knowledge base, which can be shared and unified with other sources from different health units to generate useful knowledge in the definition of strategic planning policies in the health care of the population.
Research indicates that Healthcare Professionals are of great importance for the acceptance in the introduction and use of new technologies [Ward et al. 2008]. Factors such as perceived usefulness, ease of use (or usability), social influence, attitude, facilitating conditions and training significantly impact people's behavior and the acceptance of technologies in health organizations [Aggelidis and Chatzoglou 2009]. Therefore, studies that seek to understand the attitudes, intentions of use and behavior of users in relation to new ways of working are relevant and discuss approaches and models for the introduction of technology in hospital and health services [Almeida et al. 2017].
The main research motivation is to present and discuss the importance of the end users participation in the HIS evaluation process, especially in the emergency context, where professionals develop intense activities and usually act under pressure. Identifying difficulties and usage issues before putting an HIS into organizational use, enables fixes and improvements to be made in advance, promoting success in the implementation and system acceptance. Additionally, the results obtained in the evaluation allowed identifying real problems that the participants faced on a day-to-day high, and not just potential problems predicted by the system developers.
In this paper, we evaluate the usability and intention to use of SHAVI System [de Sousa Freitas et al. 2019] in the Emergency Unit of a public hospital in southern Brazil. The system was developed to automate task management and present relevant alerts about the care of hospitalized patients, being applied to care practice. The research involved professionals and students working in the unit under study, seeking to identify difficulties and necessary improvements. In addition to identifying improvements, user participation in the process of developing and evaluating a system encourage adherence to use and understanding of the importance of the system for greater efficiency and safety in the care provided.
Thus, this study is based on the following question: What is the evaluation of the professionals of the emergency unit in relation to the usability and intention of using of iSys: Revista

Theoretical Reference
In [Christodoulakis et al. 2017] the authors investigate the barriers to IT adoption in health services. They emphasize that, despite the growing advance with information technology, its adoption in health care is very slow. The study addresses the adoption of IT in in two cases of use: Electronic Medical Records (EMRs) and Remote Patient Monitoring (RPMs). Among other barriers identified (such as economic and security), the authors point out that one of the difficulties in adopting systems refers to flaws in the design of user interaction interface. Acording to the authors, systems are designed to maintain patient data, being secondary the interface design, leading health professionals to frustrating and time-consuming interactions.
In the research by [Ajami and Bagheri-Tadi 2013] they also sought to identify barriers in the adoption of EMRs technology. Among other factors discussed in the work, the authors point problems with systems usability as a barrier in the acceptance and use of systems.
Considering the importance of a good interface design, the present work sought to evaluate the system with the final user, so that improvements can be implemented before its use. the following section presents Electronic whiteboard systems concepts and the SAHVI system.

Electronic Whiteboard Alert System
With the informatization of hospital environments, traditional boards (manual) are being replaced by digital ones [Rasmussen et al. 2010, Rasmussen andKushniruk 2013]. As a collaborative technology, the electronic whiteboards (TV, monitor or any electronic device) are placed in strategic locations in the hospitals to provide information, such as the patient's general status, bed, responsible physician, medications administered and next actions [Rasmussen et al. 2010]. In a related work [Giraldo et al. 2020], discuss nurses´perceptions about large electronic screens, named e-whiteboard, located on the nurse´s station wall. The e-whiteboard was developed with user-centerd design methodology, addressed to nursing care staff. The research proposed to explore the perceptions of nurses in relation to the implementatio of the e-whitboard system [Giraldo et al. 2020].
Currently, the activities of the nursing staff working in the emergency room of the hospital under study are organized and recorded manually (on paper) and on manual blackboards or bulletin boards, which makes it difficult to store, update and retrieve information about the services provided at the unit. To meet this demand, an electronic whiteboard system called (omitted/blind review) was modeled and developed, based on the Emergency Unit of a Hospital located in southern Brazil. The requirements gathering and system modeling phase was carried out with broad participation and involvement of health professionals who worked in the emergency at the time. The interface screens were designed in an iterative process, presented to users and updated in order to facilitate data entry and use of the system during services.
The SHAVI System manages information about the care provided in the sector through 13 modules: patient, clinic, bed, hospitalization, medical prescription, medical iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:4 evolution, nursing prescription, nursing report, nursing evolution, reminders, care plan (prescription of care), anthropometric report (vital data) and the whiteboard module that issues relevant alerts and warnings to the health team [de Sousa Freitas et al. 2019].
The electronic whiteboard module is responsible for managing the sending of alerts with relevant information about the care of hospitalized patients. For the presentation of the main events, operational data on the work performed by the health team registered in the system are analyzed at runtime. In this way, whenever a relevant situation is identified, an alert about the health status of patients is issued, so that the entire health team can view it and become aware of the fact. An example of the electronic whiteboard alerts interface is shown in Figure 1.

Figure 1. Example of Electronic Whiteboard System Alerts Interface
The first column presents general information about patients admitted to the four beds in the emergency room. The second column is intended for specific information that iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:5 must be constantly observed, such as allergies, food restrictions, among other relevant observations that the health team must be aware of when performing patient care. The third column presents dynamic information about care, which is constantly updated according to medical prescriptions and care provided by the health team. In order to make viewing easy, alerts are presented in four colors: red is for alerts that must have priority due to the urgency of care; orange represents alerts for procedures that are late; yellow for procedures that should be performed soon; and blue is fixed reminders that can be registered and configured according to the individual need of a patient. The last column, on the right, presents and manages the resting position (decubitus) of the patients, which in cases of motor disability should be changed every two hours to avoid the onset of bedsores (wounds).
The description of the system's architecture and other technical information such as data management and performance of the alert system can be found in [de Sousa Freitas et al. 2019].

Method
The research is characterized as exploratory and descriptive, with a quantitative approach, and the study site was the emergency unit of a university hospital located in southern Brazil and had as participants professionals working in the sector. The Hospital in question belongs to the public health network, being a reference for care for 31 counties that make up the regional health, especially in situations of high-risk pregnancy, urgency and emergency and outpatient care. The hospital's Emergency Room has observation beds, infirmary beds (adult and pediatric) and urgent and emergency care beds (emergency room).
This research carried out a study on the usability and user acceptance of a health information system based on Nielsen's Usability Heuristics [Nielsen 1999] and on the Technology Acceptance Model (TAM) [Venkatesh and Davis 1996]. Usability assesses how easy an interface is to use, according to: ease of learning, efficiency, ease of memorization, reduction of errors and satisfaction [Nielsen 1999]. The TAM seeks to validate perceived usefulness and ease of use, which mean, respectively, how much an individual believes that a system is useful to improve their performance in their work and how much they believe that the use of a system would require less effort [Davis et al. 1989, Lund 2001]. These two variables play an important role in TAM, as they are fundamental in decisions regarding the use of technology, that is, individuals will use a given technology if they believe that its use will result in positive results, according to the variables previously presented [Davis et al. 1989].
The study was conducted with 21 professionals and students from health area who worked in an emergency room at the Hospital, consisting of four beds for critical and semi-critical patients, who can progress to discharge, be transferred and/or where emergency measures are taken. The research took place in a real controlled environment, that is, simulating situations of use of the system, from February 2019 to February 2020. The inclusion criteria were participants who were working or who worked for any period of time in the study room. Thus, the research participants are classified into two groups: 1) people who were working in the emergency room at the time the research was conducted and 2) people who had already worked in the sector, but who were no longer at the time iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:6 of the research. It is noteworthy that because it is a teaching hospital, students, residents and professors of courses in the health area were included as participants, as long as they met the inclusion criteria.
For people in group 1, first the training was carried out and the system was made available for use (not mandatory) in the emergency room during the research period, so that professionals had the opportunity to get to know it and use it in a real environment. The system was made available on a microcomputer with Internet access and two monitors (one for system use and another for displaying alerts on the board interface). Due to the high turnover of professionals in the sector and because it is a work environment where professionals work intensely and dynamically in different work shifts, it was difficult to organize training, which required several rounds and even individual training.
For group 2, people who were no longer working in the emergency room but who knew the sector, the system was presented individually and in person through mobile devices (mobile, tablet or laptop) and a simulation of use was performed by the participant, giving them the opportunity to use and learn about the system.
For both groups, data collection was performed through interviews and post-use questionnaires in order to assess the usability and acceptance of the XXX electronic whiteboard system (omitted/blind review) in the Hospital's emergency unit. In order to carry out the evaluation, we considered the recommendations for evaluating usability and the Technology Acceptance Model [Venkatesh and Davis 1996].
The focus of the evaluation was on perceived usefulness and ease of use, seeking to identify improvements that could be applied to the System, in order to improve humancomputer interaction and, consequently, improve the adherence of its potential users. A link to the online questionnaire was made available via Google Forms for each survey participant to answer it individually. In some cases, it was applied in printed form, that is, on paper. To answer the questionnaire anonymously, each participant received an identification code, so that their name could only be identified in the Informed Consent Form.
The questionnaire was divided into 4 categories, which are: 1) User profile identification (a form for the purpose of drawing a general user profile and assessing their familiarity with the use of the system); 2) Perceived Usefulness (defined as the expectation a person has about how a particular information system can help their performance [Davis et al. 1989]; 3) Perception of ease of use and learning (ease of use would be the degree of effort expended by the user to learn and use a particular information system [Lund 2001]; and 4) Usability (Systems with good usability are systems that are easy to learn, easy to remember, maximize productivity, minimize errors and maximize satisfaction) [Lewis 1995, Filardi andTraina 2008]. The answers to the questionnaire were structured according to the Linkert scale, with five levels, being 1 = Strongly Disagree, 2 = Disagree, 3 = Indifferent (or neutral), 4 = Agree and 5 = Strongly Agree.

Results and Discussion
Questionnaires were applied to 21 participants with varied professional profiles, such as employees working in the emergency unit, professors or academics in the health area who developed internships in the study room. The small number of participants was due to the inclusion criterion, that is, having worked in the emergency room and knowing the dynamics of the work where the system will be inserted. There was no resistance regarding the acceptance of participation; the main difficulty was to select participants who knew the environment under study. Table 1 shows the sociodemographic characterization of the participating users. It is observed that most participants are female, young and with incomplete higher education, which is expected considering that it is a teaching Hospital, with a lot of health care academics in internships (Table 1).
Regarding the function performed, most participants are (or at some point were) part of the nursing team, including nursing students (47.6%), nurses (42.9%) and nursing technicians (4.8%) and medical students (4.8%). The expressive participation of Nursing is consistent with other studies that emphasize the role of these professionals in the operationalization and recording of patient information during care [Lapão 2020]. This attribution makes Nursing one of the main characters in the health digitization process [Lapão 2020]. iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:8 Regarding the frequency of use of the system, despite the system being available for use for 12 months, the majority (81%) reported that they never used the system in the room. Participants who accessed the system the most (14.3%) were probably professionals who spent more time in the Unit and had more opportunity to use it. Despite the interest and participation of academics in this study, the internships in the room are temporary and usually of short duration. Still on the use of the system, 15.4% reported that they did not have contact with the system, and these professionals were recently hired.
It is noteworthy that, to promote greater adherence to the use of the system, several suggestions for improvements were collected from the professionals during the development of the system, and that training was carried out before making it available for use. An example of a suggestion made by the users was the patient's decubitus control, so that the system alerts the team about the need to change the patient's position in the bed (as shown in the right column of figure 1).
However, the commitment of professionals to use it may have been lower due to the fact that its use was not mandatory, as it has not yet been implemented in the Hospital. In addition, there were technical difficulties during the study, such as Internet instability and the lack of integration of the system with the Electronic Medical Record used in other sectors of the Hospital. It was found that, for efficient use, the system should import registration data of patients from the other system, to avoid the need for the professional to register each new patient admitted to the Unit. These technical requirements are not met by the current version of the system and will have to be addressed in a future work.
According to [Moreno 2016] interoperability between information systems is still a challenge to be solved in the health area, from the related technological apparatus, legal and administrative issues and financial resources. Thus, it was found that the integration between the Hospital's systems is of great importance both to improve its use and to use integrated data for analysis, planning and decision-making.
In the present study, although the lack of integration between systems was cited as a fact that made its use difficult, it is understood that the time required to register a patient is small and there is a low turnover of hospitalized patients. This time would be offset by the benefits that the use of the automated system could provide, since during the period in which the survey was carried out, activities were manually noted and managed on paper sheets. Therefore, even without the integration, the benefits of using the system over the manual process is a fact.
According to [Monteiro et al. 2019], the records made in the patient's electronic medical record enable agility, systematization and security in accessing information, which has an impact on more effective care management, continuity of care plans and comprehensive health care.
Still on the identified difficulties, the organization of training was hampered by the high turnover of professionals in the Unit and also because it is a work environment where professionals work intensively (urgency/emergency care). Several rounds and even individualized training were necessary. It is important to note that the high turnover of professionals can, in part, be justified due to the dynamic internships of courses in the iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:9 health area, as well as internal relocations to maintain adequate staffing for the sector. [Martins et al. 2019] carried out a study in an emergency unit of a university hospital, which aimed to calculate turnover rates in the sector. The research discusses the possible consequences of high turnover for the quality of care provided. The authors consider that knowledge and practical experience in the sector are important factors to act safely and accurately in the emergency sector.
In the present study, it was noticed that the experience in using the computer is a factor that strongly influences the intention to use technological resources instead of manual processes. It was observed that the majority (61.9%) have basic computer use skills in their daily lives and a smaller number of participants (23.8%) are able to perform more advanced tasks, such as installing and configuring software. The minority (14.3%) reported using a computer only at work, but with difficulties.
Through a study carried out in a private hospital located in northwestern Paraná, [Pereira et al. 2020] observed that the little time for adaptation and training to use the system, as well as the lack of familiarity with informatics, are challenges for the computerization of nursing work. Also according to this research, such difficulties imply fear, stress and work overload. Such feelings can be considered barriers to the use of technological tools in health care environments. Figure 2 presents the results obtained in the questions related to perceived usefulness. Although the number of participants with advanced knowledge in the use of software is small, there was a positive result in terms of perceived usefulness. In all evaluative questions in this item, the majority of the group (blue and gray band in Figure 1) fully or partially agree that the system is useful, that it allows you to perform tasks faster, that its use would increase productivity. Participants also indicated that they would feel comfortable using the system in carrying out their duties and that they believe that it could be adopted in their work.
The interest of participants in using the system to carry out the work is consistent with the literature. [Pereira et al. 2020], which indicates good adhesion and recognition of the advantages promoted by the technological tool, when there is an understanding of its applicability and mode of use. Figure 3 presents the results obtained in the questions related to the perception of the System's ease of use and learning. In these requirements, it was observed that users positively evaluated the system, they indicated that they were satisfied with the system's ease of use, and fully or partially agree that it has screens of easy and clear interaction, that it is easy to learn, use and also that they remember how to use it. However, there were participants (14.3%) who felt nervous when using the system, and one of them (4.8%) indicated that he is often confused and would use the system more often if it were easier to use.
The feeling of nervousness regarding the use of the technological tool again refers to the lack of familiarity between health professionals and information technology resources. Including the participation of these agents in the construction of the system is an important strategy to bring technology closer to the healthcare reality in health, however iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:10  Figure 4 presents the results obtained in questions related to the usability of the system. It was found that the participants consider the information provided by the system to assist its use to be clear, useful and accurate, such as the information available in the help menu, error messages issued during use, and other documentation provided to help users complete their tasks. In relation to the system's interface screens, most indicated that they are pleasant, with an appropriate format and font size, and it is easy to find the desired information. They also consider that the system provides reliable, easy-tounderstand information and that it includes all the expected functions and capabilities. This positive evaluation made by the participants is of great importance for adherence to the use of the tool, as it is the interface that enables interaction between user-system, iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:11 acting as an intermediary in achieving the outlined objectives [Ferreira et al. 2019].
Among the objective questions, participants were asked whether a version for mobile devices (for example, cell phone or tablet) could be useful and facilitate the use of the system at the bedside. To this question everyone answered yes, it could be useful. The possibility of using computerized systems and applications on mobile devices concerns mobile health, one of the areas of information and communication technology recognized as an important resource of data management mobility and flexibility in care management, as it applies at the point of patient care [Rezende et al. 2016] At the end of the questionnaire, three non-mandatory discursive questions were presented, giving the participants the opportunity to register their opinions, criticisms and suggestions. Open Question 1 "What did you like most about the system?" was answered by eleven participants, who mentioned the following characteristics: 1) ease, accuracy iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:12 and practicality of accessing information; 2) usefulness of the alert system, especially the position change alert; 3) ease in understanding the nursing prescription, evolution and annotation, which is done manually; 4) and reduction in paper waste.
It is noteworthy that the first three items mentioned are relevant aspects to the promotion of safe and qualified care, as they contribute to accessing and communicating information, warning about care to be performed and reducing the risk of care failures. According to a study carried out in a public hospital located in southern Brazil, health professionals perceive the contributions of computerization to promote patient safety [Ferreira et al. 2019].
Ten participants answered Open Question 2 "If there is anything you could change in the system to make it better, what would it be?". The suggestions received were to insert concepts such as medications in use and integrate the system with the electronic medical record used at the Hospital. The integration of different data and access to information relevant to the supervision of the patient's clinical condition are advantages presented by the computerization of care processes [Ferreira et al. 2019 It was also suggested to expand the information presented on the electronic whiteboard. The final open-field question asked for suggestions, criticisms and compliments about the system in general. The seven participants who answered this question praised the System.
Among the contributions of the present study, the assessment with the user in a real environment is highlighted, which raises relevant aspects of usability and has the advantage over the one performed in the laboratory because it considers the weight of the cognitive load associated with the main activity [Silva 2008]. It is known that the evaluations carried out in the use environment with situations and users closer to the actual conditions of use make it possible to more clearly identify the difficulties and needs of users, allowing for more effective changes in the system [Kjeldskov and Stage 2004].
In this way, the evaluation carried out in the use environment allowed changes and improvements to the system with new functionalities and even the most appropriate definition regarding the positioning of the equipment in the room. Thus, further studies are suggested to assess the use of the system in other care contexts (sectors and/or health services), considering that care environments are dynamic, which may imply different experiences with the system and identification of other strengths and/or weaknesses.
We sought to carry out the training in the real environment of use and during the work shift. Although it allows for a closer relationship with the professional, the in loco action may have made it difficult for the participant to concentrate and learn about the use of the system, given the hectic and overloaded routine of professionals and the need to transcribe all the information from the medical record on paper to the system. Thus, as a study limitation, the need to make the training method more flexible is highlighted, which may imply how the participants assimilated the necessary information and, consequently, how to apply them in the use of the system. It should be noted that in some cases it was necessary to carry out training outside the work environment, especially for participants who no longer worked in the sector at the time of the research.

Final Considerations
This research sought to answer the following question: What is the evaluation of the professionals of the emergency unit in relation to the usability and intention of using the whiteboard system? The work involved end users, seeking to identify difficulties and necessary improvements in a whiteboard system. In addition, it aims to promoting actions that encourage adherence to use and understanding of the importance of the system for greater efficiency and safety in the care provided. Identifying and implementing these needs before putting the system into use contributes to its adoption and acceptance.
Based on the result of the evaluation, the findings are that the system has great potential for use, respects the basic principles of usability and was well evaluated in most aspects, especially those considered most important to users. It was unanimous among all interviewees in this study, and in the referenced studies, that the use of computerized systems are essential elements in the daily work of the nursing team, as well as enabling the management of information and the provision of care more safely. It is also a consensus that a system must be easy to use, learn and useful, with the participation of professionals iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 24:14 being important during the construction of the technological tool. Despite the aforementioned potential, it is noteworthy that for the implementation of the system, continuous training and the integration of the system with the Hospital's Electronic Medical Record is extremely important for effective communication between the different sectors and health professionals.
As research contributions, this study highlights the importance of user participation in the entire process of building and testing a computerized system. The greater the participation, the fewer problems will be encountered throughout the implementation process. In this sense, the development of the electronic board system had the participation of the nursing team since its conception and focused on aspects that could facilitate its use on the day to day of these professionals. We believe that our findings contribuites to the understanding the complex factors related to the implementation of health tecnology.
The evaluation presented in this paper aimed to identify and correct problems in the system before its deployment. It is important to highlight that the results presented were obtained based on the responses of participants who knew the environment of use, the emergency room. However, as the SHAVI system is not yet in official use, the participants had no experience of use in the daily practice of their activities, which may influence the validity of the research results. Therefore, it is suggested that after implantation, a new evaluation is carried out to evaluate the impact of replacing the manual process with the automated one and also its acceptance with health professionals.
As future work, we intend to study the integration of the whiteboard system with the other system used in the emergency department. In addition, Gesture-based commands are also being studied to enable touchless interaction in a sterile environment.