Accessibility of Brazilian Federal Agencies’ Mobile Apps: Requirements, Conformance and Response to Complaints

Regulatory standards and legislation have contributed substantially to improving the accessibility of governmental information systems. Adequate accessibility of mobile apps involves using appropriate processes for procuring, specifying, developing, evaluating and responding to accessibility complaints. This paper presents a study concerning the accessibility of federal agencies’ mobile apps. The study encompassed an analysis of how they specify accessibility requirements, the apps’ conformance to accessibility standards and their response to formal complaints. The investigation of requirements analysed how accessibility is considered in the procurement and specification of 24 mobile apps from the Brazilian federal government, followed by an accessibility evaluation of 22 of those apps, and an analysis of how the government agencies responded to formal complaints regarding the accessibility problems encountered in their apps. The analysis of the procurement and specification processes was done on the responses to freedom-of-information requests. Preliminary accessibility evaluations were performed using an automated evaluation tool, and formal complaints were filed on the official ombudsman service of the Brazilian Federal Government, as an analysis of the effect of social control measures over accessibility issues. The first results revealed that only seven out of 24 mobile systems included accessibility requirements in their procurement and specification. Eight agencies explicitly stated they did not include accessibility requirements. Some agencies classified the information as secret or stated they did not possess the information. All 22 mobile apps live in 2021 had accessibility problems encountered in the automated evaluation. Sixteen of the 22 mobile apps reported had responses stating the complaint was forwarded to the development team for correction, while three did not answer the complaint, and three did not respond accordingly. Elements from the Regulatory Compliance Theory provided grounds to identify low levels of enforcement of accessibility laws and low deterrence. Some agencies indicated in their responses that moral obligation Cite as: Olivera, A. C., Silva, L. F., Eler, M. M. & Freire, A. P. (2021). Accessibility of Brazilian Federal Agencies’ Mobile Apps: Requirements, Conformance and Response to Complaints (Acessibilidade dos Aplicativos Móveis de Órgãos Federais Brasileiros: Requisitos, Conformidade e Resposta a Denúncias). iSys: Revista Brasileira de Sistemas de Informação (Brazilian Journal of Information Systems), 14(4), 45-73. DOI: 10.5753/isys.2021.2016

Abstract. Regulatory standards and legislation have contributed substantially to improving the accessibility of governmental information systems. Adequate accessibility of mobile apps involves using appropriate processes for procuring, specifying, developing, evaluating and responding to accessibility complaints. This paper presents a study concerning the accessibility of federal agencies' mobile apps. The study encompassed an analysis of how they specify accessibility requirements, the apps' conformance to accessibility standards and their response to formal complaints. The investigation of requirements analysed how accessibility is considered in the procurement and specification of 24 mobile apps from the Brazilian federal government, followed by an accessibility evaluation of 22 of those apps, and an analysis of how the government agencies responded to formal complaints regarding the accessibility problems encountered in their apps. The analysis of the procurement and specification processes was done on the responses to freedom-of-information requests. Preliminary accessibility evaluations were performed using an automated evaluation tool, and formal complaints were filed on the official ombudsman service of the Brazilian Federal Government, as an analysis of the effect of social control measures over accessibility issues. The first results revealed that only seven out of 24 mobile systems included accessibility requirements in their procurement and specification. Eight agencies explicitly stated they did not include accessibility requirements. Some agencies classified the information as secret or stated they did not possess the information. All 22 mobile apps live in 2021 had accessibility problems encountered in the automated evaluation. Sixteen of the 22 mobile apps reported had responses stating the complaint was forwarded to the development team for correction, while three did not answer the complaint, and three did not respond accordingly. Elements from the Regulatory Compliance Theory provided grounds to identify low levels of enforcement of accessibility laws and low deterrence. Some agencies indicated in their responses that moral obligation

Introduction
The increase in the use of digital government information systems has brought substantial opportunities for widening access to governmental services to citizens. However, if appropriate measures are not taken to make such systems accessible to everyone, including disabled people, many citizens may have their rights refrained and limited access to essential governmental services.
As the use of mobile devices has significantly expanded in the past decade, so has the offer of governmental services via mobile apps. However, evidence has been gathered on the lack of accessibility of governmental websites Souza 2017, Sonnenberg 2020] by means of evaluating such systems according to established guidelines, and some studies have also shown that the lack of accessibility has also been an issue in mobile governmental apps [Serra et al. 2015a, Quispe and Eler 2018, Matos et al. 2021, da Silva Neto et al. 2021, Kozievitch et al. 2021].
Most previous studies on the accessibility of mobile apps and websites have focused on examining the prevalence of accessibility problems, be it by user evaluation of manual or automated inspections. However, few studies have investigated broader aspects that are important in the field of Information Systems, such as the organisational and the processes involved with the development and implementation of such systems, and how these organisational aspects impact the accessibility of government systems.
Procurement for commissioning systems developed by companies for governments and explicit inclusion of accessibility requirements for internally-developed systems are fundamental to make governmental systems more accessible [Easton 2013]. However, despite the presence of regulation on accessibility for governmental systems such as the E-government Accessibility Model (e-MAG -Modelo de Acessibilidade em Governo Eletrônico in Portuguese) [Brasil -Departamento de Governo Eletrônico 2014], there is little knowledge about the extent to which governmental agencies have included accessibility requirements when commissioning the development of their mobile systems. The case of mobile systems is particularly challenging, as current legislation is still imprecise regarding the regulation of accessibility of mobile apps, when compared to more precise definitions of accessibility of websites.
A previous study from the authors of the present study [Oliveira et al. 2020] investigated a sample of 24 mobile digital government systems from the Brazilian federal government, examining whether they included accessibility requirements in the development and procurement processes for such systems. Data collection was performed using freedom-of-information requests to the respective governmental agencies in charge of each app. The discussion of the responses provided by the agencies employed elements from the Regulatory Compliance Theory [Sutinen and Kuperan 1999] to explain possible causes of the lack of accessibility and to point ways to improve policy for mobile digital iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys government systems in Brazil.
However, previous studies have not analysed the relationship between how the specification of accessibility in procurement and requirements (or lack it) impacts the implementation of the apps available to the public. Most studies in the literature have devoted their efforts to describing the types of accessibility problems encountered in governmental systems [Oliveira and Souza 2017, Serra et al. 2015b, Corado and dos Santos 2020, Freire et al. 2009]. There is also limited understanding of how governmental agencies respond to citizens acting their social control and presenting complaints about the lack of accessibility in mobile information systems.
The present paper extends the previous study [Oliveira et al. 2020] to address those gaps. To compare the results of specification of accessibility and their implementation, the present study involved automated accessibility evaluations of the apps evaluated in the previous study by the authors. To understand the impact of social control by citizens over accessibility issues in governmental apps, the study included the analysis of the responses to formal reporting of complaints of these problems using the federal government's ombudsman service to their agencies. The study provides details about the types of problems encountered by preliminary automated accessibility evaluations on the apps and how they impact citizens, and provides a detailed discussion about the responses the agencies provided to formal complaints regarding accessibility problems in digital mobile governmental apps.
The study follows the initial discussions from the Regulatory Compliance Theory started in a previous study [Oliveira et al. 2020] to explore features of how social control of governmental organisations can help improve the surveillance and more effective implementation of accessibility in governmental information systems.
This paper aimed to answer the question "how is accessibility considered in mobile apps from the Brazilian federal government in the specification, implementation and response to complaints of lack of accessibility". To answer this research question, the following methodological aspects were employed: • The specification of requirements of mobile apps was performed by means of freedom-of-information requests to governmental agencies and analysis of documents and responses, in light of the Regulatory Compliance Theory; • The implementation of accessibility requirements was done by means of automated accessibility evaluations of the apps to verify conformance to guidelines; • The analysis of formal complaints was made by using social control instruments with online ombudsman services and analysis of the response to complaints by the agencies.
This paper is organised as follows. Section 2 presents concepts related to accessibility, freedom-of-information legislation and regulation. Section 3 presents the main methods for data collection and analysis. Section 4 presents the main results, which are discussed in Section 5. Section 6 presents conclusions and future work. Digital accessibility can be understood as the capacity of a digital product, service or content to be perceived, understood, and operated by a wide range of users, regardless of their physical, mental or intellectual disabilities [Web Accessibility Initiative 2005]. The concept of digital accessibility can be adapted from the definition from ISO 9241-171 [International Standards Organization 2008] as the degree to which disabled users can use a digital product or website to fulfill their goals with efficiency, effectiveness and satisfaction.
Digital accessibility plays an essential role in digital government initiatives since it aims to strive that all citizens have equal access to information and services provided by the government. Equal access is vital for governments to achieve goals such as to promote the interaction between citizens and government and participatory decisionmaking [Bertot et al. 2008]. Therefore, the Brazilian Federal Government have created several laws to assure that people with disabilities have the same rights as any Brazilian citizen. The rights of people with disabilities are guaranteed by the Brazilian Constitution of 1988 and by extensive legislation that regulates how physical and digital accessibility should be implemented to remove attitudinal, architectural and virtual barriers people with disabilities face daily.
When it comes to digital accessibility, Decree No. 5.296 of 2004 was the first piece of legislation that defined that the web portals and websites of all governmental agencies must be accessible to disabled people. A few years later, the Ordinance No. 3 of 2017 established that the adoption of the Accessibility Model in Electronic Government (e-MAG) [Brasil -Departamento de Governo Eletrônico 2014], from the federal government, is mandatory for federal governmental web portals. In 2015, the Brazilian government promulgated the Brazilian Law of Inclusion [Brasil 2015], which states that the websites of all governmental agencies and private organisations based or with commercial operation in Brazil must be accessible and follow guidelines from international standards.
Despite the advances in legislation, Brazilian laws are currently reticent in the regulation of the accessibility of mobile apps. The implementation of standards such as e-MAG [Brasil -Departamento de Governo Eletrônico 2014] has only been regulated for websites, with a gap for technical guidance for governmental bodies outside of the federal stance and for private bodies.

Accessibility standards and guidelines
The importance of digital accessibility for the inclusion of people with disabilities in the digital information society and the inclusiveness philosophy behind the development of the World Wide Web motivated many organisations to create a set of rules or guidelines that promote accessibility.
The World Wide Web Consortium (W3C)'s Web Accessibility Initiative (WAI) develops standards and support materials to help developers to understand and implement iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 49 accessibility. One of their most prominent contributions to web accessibility is the Web Content Accessibility Guidelines (WCAG). WCAG is a standard for web content accessibility that helps to adhere to the needs of individuals, organisations, and governments internationally [Kirkpatrick et al. 2018]. By content the WCAG means information such as text, images, and sounds; or code or markup that defines structure, presentation, etc.
The WCAG [Kirkpatrick et al. 2018] splits recommendations into four principles, that content should be: perceivable (users must be able to perceive information and visual components); operable (users must be able to interact with the application); understandable (users must be able to understand the information as well as the operation of the user interface); and robust (users and assistive technologies must be able to interpret content in a reliable way). All WCAG recommendations have success criteria (SC), which define whether one specific guideline has been implemented. All criteria are split into three levels of conformance: A, AA and AAA, in which SC A are easier to accomplish while SC AAA are harder to achieve.
The Brazilian Accessibility Model in Electronic Government (e-MAG) [Brasil -Departamento de Governo Eletrônico 2014] is highly inspired by the WCAG standard [Kirkpatrick et al. 2018]. This Brazilian model is intended to guide the development and the adaptation of federal government digital content, enabling users to have access to information and services [Brasil -Departamento de Governo Eletrônico 2014]. It has forty five accessibility recommendations split into six sections: marking, behaviour, content/information, presentation/design, multimedia, and forms. Unlike WCAG, the e-MAG model does not set conformance levels.
Most accessibility standards and guidelines are focused on web content accessibility. With the technological shift to mobile platforms, however, mobile accessibility has become of interest. Although many aspects of mobile accessibility are covered or can be adapted from web accessibility guidelines, there have been many initiatives towards defining specific guidelines for mobile apps.
The WCAG 2.1 [Kirkpatrick et al. 2018], the newest version of the standard, includes recommendations for handling barriers that can affect users with disabilities on mobile devices. The British Broadcasting Corporation (BBC), a public service broadcaster in the United Kingdom, has created the BBC Mobile Accessibility Guidelines [British Broadcasting Corporation (BBC) 2017]. BBC's accessibility recommendations are very comprehensive and are split into individual topics: audio and video, editorial, design, focus, forms, images, links, notifications, images, scripts and dynamic content, text alternatives and structure. e-MAG does not include mobile accessibility recommendations explicitly, but there have been initiatives to propose adaptations to mobile devices [Quispe and Eler 2018].

Access to Information Law
Straightening the relationship with citizens is one of the main goals of digital government initiatives. Besides accessibility, which assures all citizens can access information/services and engage in participatory decision-making, transparency is a crucial element in this context. In a broad sense, transparency policies define how much access to iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys public government information citizens are qualified to, and which actions citizens should take in order to have access to information. Besides, transparency policies should define how citizens should proceed when governmental agencies deny their rights to have access to public information. Governmental agencies can promote active transparency by voluntarily publishing important information on their websites or official platforms. As many pieces of information may concern specific citizens or organisations, many public organisations only disclose information by freedom-of-information requests.
The Brazilian Federal Government promulgated the Law nº 12.527/2011 [Brasil 2011] to regulate the right of any person to request and receive information created or maintained by any public institution. This law is known as the Law of Access to Information (LAI). There are some restrictions to the access to information that could put the society of the nation at risk. Citizens can request information from any public institution using the Information Services to Citizens (SIC, in Portuguese). An electronic version of this service has already been created (e-SIC). All information requested must be immediately delivered when they are available. In any other case, the public institution has 20 days to deliver the requested information. This deadline can be extended in 10 days when there are important reasons for the delay.
The data collection procedure for this research was performed using e-SIC as the main source for requesting information. Section 3.2 presents the details of the requests filed for the data collection.

Social Control of Governmental Organisations
The term "Social Control" (Controle Social) has been used in the context of Public Administration research in Brazil to refer to the involvement of citizens and society in the oversight and decision-making in governmental actions [Silva 2002]. Controladoria Geral da União (Federal General Controller) [Controladoria Geral da União 2021] defines social control as "the effective participation of society in the surveillance of the application of public funds and of the conception and oversight of public policies". According to the surveillance agency, effective social control is essential to consolidate democracy and social participation.
Social control has the potential to involve a more diverse set of citizens in public matters, bring more openness, efficiency and democratic views to public management, aiming at the common good [Controladoria Geral da União 2021]. In Brazil, social control can be exerted in several forms, including traditional offline media such as public assemblies, open committees at different levels, and ombudsman services for formal complaints. The availability of digital participation services and digital official ombudsman services is a significant advancement to provide citizens with tools for social control. With digital ombudsman services, the filing of complaints may be done more efficiently, regardless of geographical distance, and feedback may be provided in a timely manner [Santos et al. 2019].
Ensuring that citizens have appropriate feedback on their complaints in ombudsman services is essential to promote the willingness to engage in social control. In the context of this paper, social control was analysed as an instrument to file complaints iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys about the accessibility of mobile apps that provide public services using digital ombudsman services. Analysing the type of responses by government agencies is a critical aspect to understand how public agencies perceive digital accessibility as their responsibility and how citizen engagement could promote digital accessibility by more active social control.

Systems Acquisition and Implementation in Brazilian Federal Institutions
Digital government initiatives depend on digital products, services and portals. Many digital solutions are not developed in-house due to high demands and the limited number of specialised employees in many public agencies. In that case, public procurement is conducted so private companies can implement solutions demanded by the government. Agencies of the federal government must follow many rules when developing digital services or products. Accordingly, the same rules should apply for third-party contractors.
In that sense, the Brazilian Normative Instruction 04/2014 (IN 04/2014) [Brasil -Secretaria de Logística e Tecnologia da Informação 2014] specifies the process that federal agencies should follow to purchase Information and Technology Solutions from third-party companies and for in-house development. Specifically, this piece of legislation defines that a preliminary technical study must be conducted by the technical staff and the solicitors at the government side to specify business and functional requirements based on, among others: stakeholders demands, existing solutions in the market, existing solutions in other federal agencies, and digital government standards such as the Brazilian Accessibility Model in Electronic Government (e-MAG).
The instructions present in IN 04/2014 show the importance of observing accessibility requirements in contracts and procurement procedures when developing systems for government bodies.

Regulatory Compliance Theory
From the replies obtained from Brazilian federal government bodies in freedom information requests, some justifications as to whether they requested accessibility requirements or not provided some initial insights as to reasons for adhering or not to accessibility legislation concerning information systems. In order to better understand these issues raised in some comments in the responses and to pave the way for future research to deepen the understanding of such factors, elements from the Regulatory Compliance Theory (RCT) were analysed in this research. The main concepts of the RCT model defined by Sutinen & Kuperan [Sutinen and Kuperan 1999] are described.
Mitchell [Mitchell 1994] defines regulatory compliance as "the conformity of an organisation's behaviour with a prescribed legal rule". According to his definition, "noncompliance is thus a deviation from prescribed legal behaviour". According to Sutinen & Kuperan [Sutinen and Kuperan 1999], compliance behaviour is determined by three main factors: 1) Deterrence (expected penalty), 2) Illegal gains, and 3) Moral Obligation and Social influence.
The concept of deterrence is linked to the action of discouraging an action by showing possible consequences, especially penalties. In Sutinen & Kuperan's model [Sutinen and Kuperan 1999], this concept is linked to effective law enforcement, in terms of detection and applying sanctions to nonconformants. iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys In the economic model of regulatory compliance, Sutinen & Kuperan [Sutinen and Kuperan 1999], in the context of illegal fishing, explain that "if the typical illegal gains are greater than the gains from legal fishing, the expected penalty should be large enough to offset the difference between legal and illegal gains".
In Sutinen & Kuperan's model [Sutinen and Kuperan 1999], Moral Obligation and Social influence have two main related concepts: Legitimacy and Moral Development and Personal Values.
Legitimacy of authority involves four main factors. Effectiveness of the outcome involves the extent to which the effect of the compliance may be observable. The distributive justice of the outcome concerns "the perceived fairness of how the benefits or sacrifices are shared among the affected parties" [Sutinen and Kuperan 1999]. The efficiency of the process involves "the speed and efficiency with which people perceive the authority responding to problems within the scope of the authority's jurisdiction" [Sutinen and Kuperan 1999]. The procedural justice involves "how fairly the authority treats people and the concerns of those affected by the process" [Sutinen and Kuperan 1999].
Moral development and personal values are linked to the importance of people's personal characteristics as determinants of compliance behaviour. In this concept, "information functions as a form of moral suasion, influencing personal values, which in turn influence behavior" [Sutinen and Kuperan 1999].

Research method
This investigation sought to analyse whether Brazilian mobile apps from federal agencies had accessibility requirements, basic conformance to standards, and how the agencies respond to official complaints in ombudsman services.
The following sections present the selection of mobile applications, requests for requirements and procurement processes, automated accessibility evaluation of the applications and filing complaints for non-conformant mobile apps.

Selected mobile applications
We selected only mobile applications developed in the context of the federal government since most pieces of legislation are explicitly mandatory for federal agencies. We also decided to target only mobile applications developed for the Android platform since it is the most common operating system used in Brazil [StatCounter 2019]. Based on such decisions, we used the Google Play Store search mechanisms to find mobile applications in the list of applications related to Citizenry and Law and Public services. From this list, we filtered out applications that did not belong to the Brazilian federal government, such as state-level apps and private apps with public ends. By the time we conducted this search (7th August 2019), 24 mobile applications were included in the research. In September 2019, the federal government created an official website called gov.br to group all services, including mobile applications developed by the federal union. We did not update our list of investigated apps to enable a longitudinal study on the same sample of apps, considering the responses given in the first round concerning the requirements, and following the automated accessibility evaluations and formal complaints on the ombudsman service.

Requests for Requirements and Procurement Processes
We used the Electronic Information Services to Citizens (e-SIC, as described in Section 2.3) to request federal agencies information about the procurement process and the details on the development of specific mobile applications, be it in-house or outsourced. Table 1 shows the federal agencies and corresponding mobile applications for which we requested information.
Even though the Brazilian Law of Access to Information does not require the citizens to provide a reason to request any information, we wrote a text to explain that we intended to use the information requested to conduct academic research. For each mobile application, we requested the following items: • Technical specification of the procurement, cooperation agreement, or service request. • The contract with third-party contractors (if it was the case).
• Requirements Document and/or business rules elicited and registered for the software development. • Information on the technical staff responsible for the development of the mobile app. • Documents concerning the system certification and validation process. • Accessibility requirements specified for the system (if any). • How accessibility requirements were validated (if any).
Some federal agencies claimed that accessibility requirements were secret. Hence, we prepared the following message to appeal in such cases: "Accessibility requirements iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys There might also be cases in which the federal agency transferred the responsibility to the third-party contractor or to other agencies that developed the mobile application. In the case of federal agencies that develop IT systems, we requested the same information to the organ responsible for the system.
All data gathered from the answers to the freedom-of-information requests were tabulated, recording the date, answer to each question, and possible new requests of information or appeals in case of denial of access. We also recorded specific information according to each specific question, such as who implemented each system, technical specification (call for bids, cooperation agreement, service order), contract (in case of outsourcing), requirements documents, development team, ratification and acceptance, and other details of standards and legal aspects concerning accessibility that were described in the documents.
In the cases in which requirement documents, contracts or procurement calls for bids (editais de licitação) were provided, we performed an analysis on such documents to analyse whether accessibility requirements and verification procedures had been included.

Automated Accessibility Evaluation of Mobile Apps
Automated evaluations were performed using the tool Accessibility Scanner 1 . The tool is able to scan and detect a selection of accessibility problems in Android apps automatically.
The evaluation is performed using the button "Accessibility scanner". Results can be viewed as a report and can also be forwarded to app developers via the Google Play Platform.
The mobile apps evaluated were the same analysed in the the freedom-ofinformation requests described in Section 3.1. Only two apps were not available at the time and were not evaluated (Sisu and Anatel Serviço Móvel). This evaluation was performed on public screens that did not need login.

Filing Complaints on Ombudsman Services
All apps evaluated had accessibility problems encountered in the evaluation. After the evaluations, we used the official ombudsman service from the Brazilian Federal Government, available at the Platform FalaBr -Plataforma Integrada de Ouvidoria e Acessoà Informação do Governo Brasileiro 2 .
Complaints were filed on 31st January 2021, containing the name of the app and the list of problems encountered, and mention of Brazilian legislation regarding the accessibility of digital services.
Brazilian Law 13.460/2017 states that governmental agencies have up to 60 days to respond to formal complaints recorded in their ombudsman services.

Results
This section presents the main results obtained from this study, including general data about the answers from government agencies on the accessibility requirements of their mobile information systems, the process of freedom-of-information requests, institutions in charge of the implementation of the systems and provision of supporting documents regarding the implementation, accessibility problems encountered in the automated evaluations, and responses to the complaints filed in the ombudsman services.

General information about the responses and request for requirements and procurement information
All initial requests were sent to the respective agencies responsible for each system on 16th August 2019. Three requests (Meu INSS-Serviço, CTPS Digital and Sine Fácil) sent to DataPrev (an IT agency that develops systems for pension-related matters) replied that they only provided services to other agencies, and that they could not provide information. New requests were made on 11th September 2019 to each agency pointed by them.
The apps Bolsa Família, Caixa Trabalhador and FGTS had responses from the Ministry of Economy alleging "commercial secrecy" on the more detailed requests, including details about accessibility requirements. The agency maintained their stance on secrecy, despite the arguments provided.
The app FAB -Força Aérea Brasileira was developed in house by the National Airforces. However, the agency replied that the available documentation about the development did not have enough data to inform if accessibility requirements had been requested.
After having affirmed that the accessibility requirements of the apps CNPJ, MEI, Meu Imposto de Renda, Pessoa Física and Viajantes were classified as secret, in response to the appeal, the Ministry of Economy provided an answer affirming that companies should follow internal interface standards that included accessibility.
Information about the apps SNE Denatran and Carteira Digital de Trânsito, under the responsibility of the Ministry of Infrastructure, had provided no answer to date. The agency alleged that SERPRO, the federal data processing agency, would have the data, but SERPRO claimed they were not able to provide the information.
The app Sinesp Cidadão, from the Ministry of Justice and Public Security, still does not have information about accessibility requirements. The Ministry claimed that only SERPRO would have the information, but the agency declared that this information was considered "commercial secrecy", and that SERPRO does not answer to freedom-ofinformation requests linked to academic research.  requests resulted in allegations of information secrecy, with a subsequent appeal from the researchers. The reasons for appeals and further requests are detailed as follows.
Agencies denied access to the information due to commercial secrecy and national security. Even though there could be specific aspects in the requirements documents of some of the systems, we appealed on the grounds that accessibility requirements in the systems would not be bound by secrecy allowances, since they are a legal requirement. In such cases, we requested that they answered at least whether the accessibility requirements had been requested.
Other agencies answered that they did not possess the information, and pointed other agencies that could provide the information. In such cases, we registered a new request to the other agency. iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys Table 3 presents a list of companies and government agencies who were pointed as the responsible for the development of the mobile apps investigated in this research. Among the private companies who were commissioned to develop the systems, CAPGEMINI had the largest number of systems, being in charge of 4 apps. From the government agencies, SERPRO and the IT department (COTEC) from the Ministry of Economy were in charge for 5 apps each.  Table 4 presents a summary of the provision of the requested documents to perform the analysis, being the technical specification, contract, requirements and ratification process.

Provision of technical specification, contract, requirements and ratification procedures
Regarding the technical specification for development, only 12 out of 24 (50%) of the systems had their documents provided, with others being classified as secret (7)  staggering number of 12 out of 24 (50%) had their requirements documents classified as secret. Only 7 full requirements documents were provided.
The documents provided included call for bids (for systems that were outsourced), cooperation agreements, when the system was developed by another governmental agency, or service orders, when it was developed internally. Requirements documents included different types of documents, from full detailed specifications to more succinct specific accessibility requirements, and others with external documents with internal design guides pointing to accessibility requirements.
The description of the development team and ratification processes for the systems were also classified by 10 out of 24 systems (41.7%) as secret. In some cases, appealing to obtain at least information about accessibility requirements resulted in obtaining more superficial details about accessibility, but not the full documents.
It is also worthy noting that 5 systems did not have information provided by the responsible agencies about the development team. This shows that outsourcing has very little overseeing by the contracting governmental agencies. Table 5 shows the types of answers provided for the questions regarding the presence of accessibility requirements in the specification of the mobile apps investigated and whether the agencies specified how accessibility should be verified.

Accessibility requirements in the development and verification of mobile apps
Only 7 out of 24 apps (29.2%) had accessibility requirements included in their specification -CNPJ, MEI, Meu Imposto de Renda, Pessoa Física, Viajantes Sigepe Mobile, Planalto Legis. In a preliminary verification of such systems performed by means of an inspection using the TalkBack screen reader on Android systems, these systems in fact showed some evidence of effort on improving the accessibility, with labelling of buttons with figures and navigation via gestures. However, even in these apps, we still encountered evidence of some problems, such as occasional buttons without labels that could be read out by screen readers.  Agencies responsible for four of 24 apps declared secrecy on the accessibility requirements of their apps, even after all the appeals filed up to the date this paper was written (Bolsa Família Caixa, Caixa Trabalhador, FGTS and Sinesp CIdadão). Agencies in charge of two apps (FAB -Força Aérea Brasileira and CTPS Digital) stated they did not possess the information, whilst three other apps ended up no providing the requested information, mainly due to successive forwarding of responsibility between different agencies (Meu INSS -Central de Serviços, SNE Denatran and Carteira Digital de Trânsito).
With regards to defining verification procedures, the results showed that only two out of 24 (8.3%) of the agencies specified procedures for the verification of the accessibility requirements (Sigepe Mobile and Planalto Legis). Five of the seven apps that had accessibility requirements specified had no verification procedures for certification of the deployed systems.
The answer for the app Planalto Legis was provided directly by the Secretary of the Presidency. The answer include a text stating that "the requirement stated that the app should be in accordance with accessibility standards, enabling disabled users to use it". Regarding the verification, they stated that "verification was performed by means of tests with users. For example, the colours of the app were changed after a user with Daltonism reported visualisation difficulties".
After denials to provide information about accessibility requirements, the Ministry of Economy provided information about five apps: CNPJ, MEI, Meu Imposto de Renda, Pessoa Física and Viajantes. They stated that the companies hired to develop these systems were asked to follow interface standards written in Portaria No. 702, 3rd May 2016, which included accessibility requirements.
The Ministry of Health, in response to the request about the Meu DigiSUS app, replied that they did not include accessibility requirements for the app, because "the app uses [accessibility] resources from the [mobile] device itself". However, they did not point out any efforts made to make the app itself compatible with the accessibility re-iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys sources available in mobile apps, such as screen readers as TalkBack for Android. If such adjustments are nor appropriately made, the accessibility resources from the device will not work properly.
The Ministry of Education provided detailed information about the user stories and requirements. However, careful analysis of the documents found no evidence of accessibility requirements in systems such as SISU and ENEM, even though they are used by millions of students across the country, which include a substantial number of disabled students.
The Ministry of Infrastructure justified that the app ANEEL Consumidor did not have any accessibility requirement requested because "The government's accessibility standards (e-MAG and e-PING) are only mandatory for web applications. The app Ouvidoria [(Ombudsman)], for example, does not have high contrast and increasing font features".

Automated evaluations of Governmental Apps
In the beginning of 2021, we performed an automated evaluation of the apps listed in the previous study [Oliveira et al. 2020] that were still live.
Automated evaluations are not able to cover all types of accessibility problems that people with disabilities would encounter in digital applications. However, the presence of problems found by automated tools show that even the most basic problems have not been examined. Table 6 presents a list of the types of accessibility problems encountered in the apps, with the number of apps that had that problem, the number of instances, and the average number of instances per app. The number of instances corresponds to how many times a given violation of a guideline was verified in the evaluation. Of the 24 apps that were surveyed in the initial study to request requirements, specifications and procurement requests, 22 were still live in the beginning of 2021. All 22 apps had at least one of the problems that were evaluated by the tool Accessibility Scanner.
The problem "Touch area too small" had the highest average number of instances, occurring in all 22 apps evaluated. This problem corresponds to small elements iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys

Figure 1. Example of small touch area in the app Carteira de Trânsito Digital
that may be too difficult to tap for people with motor disabilities, low vision and low precision, such as older adults.
One of the apps that had such problem was the Carteira Digital de Trânsito, from the Ministério de Infraestrutura. Figure 1 shows an example of such problem. In that screen, the text of the link Termo de Responsabilidade had 24dp in height, with a recommendation of having 48dp or more.
The problem "Colour contrast in text" was also encountered in all apps examined, and also had a high number of instances. In the app Carteira de Trânsito Digital, there was a high number of instances of such problems. Figure 2 shows an example with several excerpts of text with light grey on white, which could be difficult for people with low vision or elderly users. The ratio between the luminosity of the background and front colours are not enough to allow for clear reading.
The problem "Lack of textual labels" still occurred in eight apps. This is considered one of the most basic accessibility problems. If textual descriptions are not included in graphical elements, screen readers are not able to convey any information to blind users. Figure 3 shows an example from the app Anatel Serviço Digital, in which several elements, including the hamburger menu, do not have any textual labels, preventing the interaction of blind users.
The problem "Inappropriate textual description" had 129 in three different apps (Viajantes, Sinesp Cidadão and Anatel Serviço Móvel). Figure 3 shows an example from iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys   The problem with "Clickable items not accessible by other means" occurred once in the app Receita Federal -Pessoa Física from the Ministry of Economy. There were elements that could not be activated by alternative means to click, such as gestures used by screen readers. Figure 4 shows the elements (back arrow and current location on map). These elements can only be tapped visually. When blind people use gestures to "swipe right" to reach the next element, for example, this element cannot be focused.

Responses to formal complaints for accessibility problems
All complaints to the federal agencies in charge of the apps examined were made on the 31st January 2021. Considering current legislation, they would have up to 60 days to answer the complaint.
The results of the requests were verified on 31st March 2021, when the deadline was reached. Table 7 presents the status of all the requests up to that date. The analyses showed that 72.7% of the requests were sent to the IT team to fix the problems. Unlike iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys with the responses to the freedom-of-information requests, there was a more positive response to the complaints. Only one answer was classified as insufficient, stating that they acknowledged they received the complaint. In such cases, agencies did not inform any procedures that would be taken. One response was not compatible. The agency recommended that the complaint be made in person.
The General Secretary of the Presidency (SGPR -Secretaria-Geral da Presidência da República) provided an inadequate answer regarding the app SIGEPEmobile. They stated that the ombudsman service was not the adequate place to formalise the complaint. This is in serious breach with current legislation in Brazil. An appeal has been filed. However, no response had been provided until the day this paper was written. Following is a free translation of the response.
In attention to your message, we clarify that the Ombudsman service is not the adequate channel for your request. We inform that you should address the Secretary of Management and Human Resources Performance -SGP/SEDGG/Ministery of Economy. Address: Esplanada dos Ministérios, Bloco C, 7º andar -Gabinete.
The message makes it clear that the governmental agency tried to escape from an official obligation, suggesting that a much more bureaucratic and costly means be used to file the complaint.
Ministério da Infraestrutura and SERPRO -Serviço Federal de Processamento de Dados also provided an inadequate answer, stating they could not forward the complaint regarding Carteira Digital de Trânsito.
DataPREV provided an incoherent answer, with a standard template message explaining the procedure to change citizens' addresses in their system, regarding the complaint about SINE Fácil.
Despite the cases of lack of response in the legal time, and the inconsistent answers, more than 70% of the agencies forwarded the complaints to their IT departments of companies in charge of developing their apps.
Some agencies even showed positive willingness to perform changes and replied showing that they see the reports as a positive contribution to their work. The National iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys Electricity Agency (Agência Nacional de Energia Elétrica (ANEEL)), in charge of the app ANEEL Consumidor provided the following response (in free translation): The Ombudsman service of the National Electricity Agency (ANEEL) thanks your contact and highlights the importance of your manifestation as an instrument to improve the public services and to exert the social control of the acts made by public agents. We inform that we receive your manifestation with satisfaction and take your complaints into consideration. Manifestations like yours are an important incentive and encourages us to improve the quality of the services of consumers of electricity.
They also informed that they had performed changes in their app, following suggestions they observed on Google Play Store. This shows that the agency sought to engage with their costumers in other media too. Participation of users with disabilities in such reviews may be a good opportunity to engage with governmental agencies and to show them the accessibility aspects that need to be addressed.

The lack of accessibility requirements in mobile government information systems and regulation aspects
The results obtained in the present study are very worrisome in terms of how federal agencies have dealt with accessibility requirements. As pointed out by Lazar et al. [Lazar et al. 2015], establishing appropriate law enforcement and oversight procedures is essential to make accessibility laws effective.
However, the results have shown that a significant number of federal agencies have admitted to not including accessibility requirements for their systems.
Brazilian legislation has important landmarks in terms of regulation of accessibility of digital services for federal governmental systems, including e-MAG [Brasil -Departamento de Governo Eletrônico 2014] [Brasil 2015]. However, it seems like law enforcement and regulation of specific aspects of the law still lacks in the country.
Observing aspects of the Regulatory Compliance Theory, described in Section 2.6, we observed that the justifications shown some of the agencies, stating that accessibility to mobile systems is not mandatory, and the way they state that such requirements were not included, are indications that there is little effective in the deterrence and enforcement mechanisms in place in Brazil.
In fact, the wording of the Brazilian Law of Inclusion [Brasil 2015] needs rewording to ensure that technologies other than websites also have to be accessible.
However, governmental agencies seem to use the lack of precision in some laws to bypass legal obligations in documents such as IN 04/2014 [Brasil -Secretaria de Logística e Tecnologia da Informação 2014], which guides the iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys process of implementation and acquisition of IT solutions in government. On the other hand, IN 04/2014 also needs further improvements, as many agencies have failed to specify ways of verifying accessibility requirements, which is essential to ensure that specific requirements have been met.
Effective law enforcement by fiscalisation and sanctions is also necessary to make governmental agencies adhere to legislation and technical guides. Although the concept of "illegal gains" in RCT is not directly applicable to governments, there is an imbalance between the costs the agencies would save by not specifying accessibility requirements and the almost null costs of sanctions.
Pieces of evidence from some agencies also showed that moral obligation to disabled people appeared to be an important aspect in some agencies that reported to request accessibility requirements and evaluations with users in their systems. This way, it is important that awareness-raising campaigns with civil servants and developers to have more people with moral perception of the benefits of adhering to accessibility regulation in mobile systems.
Unfortunately, due to the lack of information or denial to disclose the information, it was not possible to establish whether there were differences in the specification of accessibility requirements developed by governmental IT departments and by hired private companies. There were cases with lack of specification of accessibility requirements in both cases. It is important to have appropriate accessibility requirements in both situations. In procurement processes, the precise specification of accessibility requirements is essential to ensure that government agencies can demand accessibility to outsourced systems.

Secrecy in the disclosure of official information systems requirements
One element in the methodological design of the present study made us seriously concerned. We had not expected at all that such a high number of agencies would place the requirements documents and technical details of governmental mobile information systems as secret information that cannot be opened to citizens. This is a serious threat to the possibility of citizens to oversee and watch governmental systems and their technical efficacy. Not opening systems specifications is also a threat to the transparency of such information systems and to the business rules implemented in them. Considering that the Law of Access to Information [Brasil 2011] in Brazil is relatively new (from 2011), this behaviour of secrecy on requirements documents of information systems and its implications on the transparency of public systems needs to be further investigated by the Brazilian Information Systems community.

The lack of accessibility requirements and the impact on disabled citizens
Another point that is worth discussing is the impact that the systems without accessibility requirements have on people with disabilities. Some of the mobile systems whose responsible agencies have admitted to not including accessibility requirements correspond to basic services offered by government to a range of citizens. iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys Among those systems, it is appalling that systems such as Meu DigiSUS do not have accessibility requirements, since it corresponds to a mobile system to give support to basic public health services. Further to this, the response given by the agency that the app uses "the device's accessibility resources" shows how ill-informed the managers are about accessibility needs. If apps are not developed following accessibility requirements, accessibility resources such as screen readers in mobile devices have no way to work at all.
Other systems have severe impact on citizens, such as preventing high-school disabled students from having access to educational services such as the national exam ENEM and the university application system SISU.
The accessibility evaluation performed with the automated tool also revealed that, in fact, the apps did present accessibility problems. The effectiveness of automated evaluation is limited, as automated tools cannot find all problems that would be found by evaluations by experts or with users with disabilities [Vigo et al. 2013, Power et al. 2011]. However, this means that the mobile apps potentially have more problems than the problems that were reported in this paper. This shows the need to improve the requirements elicitation and regulation, as there is evidence of accessibility problems in the app.
The results are in line with findings from previous studies in Brazil and in other countries, which also found problems on governmental websites Souza 2017, Sonnenberg 2020] and mobile apps [Serra et al. 2015a, Quispe and Eler 2018, Matos et al. 2021, da Silva Neto et al. 2021, Kozievitch et al. 2021]. Recent studies have reinforced the need to promote awareness about accessibility, especially in the mobile context [Leite et al. 2021].
Those issues show how urgent it is to incorporate more effective policies to promote accessibility in a range of technological approaches to providing digital government services to citizens, be it on mobile devices, websites or other media.

The importance of the social control of government behaviour toward digital accessibility
The results from the freedom-of-information requests showed that Brazil still has a long road ahead to having effective information access to its citizens. The analysis of procurement process and requirements specification of mobile apps to analyse accessibility aspects was very limited.
However, the requests for changes in the apps in the form of formal complaints in ombudsman services had a more positive response, with more than 70% of the agencies providing feedback and forwarding the issues to their IT departments and companies that develop the apps.
It is still an open question whether the changes will be appropriately implemented. Nevertheless, it shows the importance of more active participants from citizens in the oversight of the accessibility of digital services provided by governmental agencies.
A previous study [Siqueira and Freire 2020] showed that very few complaints about the accessibility of digital services had been filed in official ombudsman services iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys until 2019. The results from the present study shows that more involvement from society is needed to exert social control of governmental organisations and to improve the accessibility of digital services.
Whilst it is still unknown whether such actions will result in effective implementation of these requirements, the study showed that using official channels is an effective medium to raise awareness and to take accessibility out of invisibility. Our results concur with previous efforts to improve social control [Correa Vuolo et al. 2021].

Conclusions and Future Work
This paper aimed to answer the question "how is accessibility considered in mobile apps from the Brazilian federal government in the specification, implementation and response to complaints of lack of accessibility". The study counted on an analysis of a sample of 24 mobile information systems from the federal Brazilian government, to investigate whether they included accessibility requirements for their development, to evaluate their accessibility using automated evaluation tools, and to analyse the response from official agencies after presenting formal complaints in ombudsman services.
The paper presents a relevant contribution to the field of Information Systems, by analyzing systemic and organisational aspects related to the accessibility of governmental apps for people with disabilities. The research reached beyond the technical evaluation of the accessibility of mobile apps, and presented organisational and regulatory aspects. The paper presented aspects from the Regulatory Compliance Theory and evidence of how the social control by society can impact the accessibility of mobile apps.
Data were gathered by means of freedom-of-information requests, which were analysed and consolidated. Mobile apps that were still live in 2021 were evaluated using the Accessibility Scanner tool, followed by formal complaints about problems encountered on the federal government's ombudsman service.
The results from the freedom-of-information requests showed that only seven out of 24 apps had accessibility requirements included either in procurement processes to private companies, in-house development or development by other governmental agencies.
Accessibility evaluations showed that all 22 apps that were still live had accessibility problems related to colour contrast, lack of textual labels, inadequate textual descriptions, inaccessible controls and clickable elements that were too small.
Responses from the complaints filed on the ombudsman service showed that more than 70% of the agencies replied positively, forwarding the complaints to their IT departments or contractors to solve the issues. However, other agencies still failed to respond or provided inappropriate answers.
Evidence in the justification sent by some agencies for not adhering to accessibility requirements resonated with elements of the Regulatory Compliance Theory. The analysis indicated that their low deterrence and law enforcement in current accessibility policies to governmental information systems in Brazil, as well as an imbalance in the purported costs that would be saved by ignoring such requirements and the lack of sanctions. Moral obligation was still shown as an important factor to be fostered with agencies, from iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys 70 agencies that reported believing on the importance of the inclusion of disabled people.
The study had difficulties with the high number of agencies that placed requirements documents of mobile information systems as secret information. This raises questions to the IS research community in Brazil as to the consequences of secrecy of technical documents to IS transparency and fiscalisation.
The study also had limitations in its scope and on the reach of the methods. The number of apps evaluated was still limited. The federal government has released more apps in recent years. Future studies could broaden the studies by investigating a broader range of apps. Some questions related to Regulatory Compliance Theory cannot be addressed with the method employed. In-depth research with interviews, for example, could provide more details about the behaviour of different stakeholders. Social control could also be further analysed by observing long-term effects of the effective engagement of citizens in the surveillance of the accessibility, and not only by complaints filed by researchers.
As future work, we intend to expand the analysis of more recently published mobile apps by the Brazilian federal government, as well as to extend the study to other levels of state and municipal governments. We also intend to perform longitudinal analyses of mobile apps of governmental agencies that received formal complaints and to analyse the effect these complaints had on the behaviour of those agencies and in the deterrence aspects.