Comparison and Choice of Computational Architectures Based on Cost-Value Approach

Software engineers make use of several computational architectures (CA) to host an application, such as desktop, web, and cloud computing architectures. As the requirements vary according to the desired CA, developers may face two problems: determining which requirement better fulfills a CA and determining which CA fulfills a given set of requirements. This paper presents a new approach based on the Cost-Value Approach (CVA). We have slightly modified the CVA method (SCVA method) and also developed a new method for choosing the most appropriate CA (MMACA method). Our results provide a set of requirements ordered by priority for each CA. Finally, we discuss the current and most appropriate CA for a real project solution.


Introduction
Requirements describe information about features, services, functionalities, and restrictions of a software system [Bourque andFairley 2014, van Lamsweerde 2009]. In the software industry, a requirement is defined as a formal definition of a system function, a high-level service description, or a system restriction [van Lamsweerde 2009]. How to meet stakeholders' needs and how to manage information during a software project are fundamental issues in the area [Olsson et al. 2019]. In recent years, requirements have become increasingly dynamic and heterogeneous due to the expansion of the web and social media [Zampoglou et al. 2016].
Requirements can be classified into functional and non-functional. Functional requirements are functions that a system might perform. For instance, in the academic domain, a secretary can list all undergraduate students, and a lecturer can book a laboratory. Non-functional requirements define behavioral conditions, constraints, or quality attributes for software. For instance, non-functional requirements can be the availability of a system, i.e., 99% available on the web, and database queries should take less than three seconds to perform [Bourque andFairley 2014, van Lamsweerde 2009]. A successful information system is the one in which all software requirements are fulfilled [Belout andGauvreau 2004, Kaur andAggrawal 2013]. However, this is not a trivial task.
Whereas software components fulfill almost all functional requirements, some non-functional requirements are dependent on the computational architecture (CA) that hosts the information system. CAs are models for organizing physical and logical resources, providing a standard of development process [Patterson and Hennessy 2013]. Examples of CAs are desktop, web, cloud, fog, and mobile. Given this diversity, the choice of the most appropriate CA may be complex because it depends on a software requirement, such as availability, scalability, access mode, among others [Bessa et al. 2016].
Although similar, computational architectures and software architectures are different concepts. Computational architectures refer to the infrastructure (e.g., operating system, network platform, data storage, among others) in which the system or application is hosted. Software architectures concern system design decisions, software elements, and how these elements relate to each other [Taylor et al. 2010]. A computational architecture can host software with different software architectures because a computational architecture may be more suitable for a given software architecture standard.
The requirements engineering (RE) process comprises a set of activities to discover, analyze, validate, and manage a system requirement.
Authors in [van Lamsweerde 2009] report that there is no single RE process suitable for all organizations. Moreover, they state that a generic process consists of: deciding whether the system is useful (feasibility study); discovering, ranking, and negotiating requirements (elicitation and analysis step); organizing the requirements to be implemented (specification step); and, checking whether the requirements fulfill the customer's desires (validation step). RE is one of the most important activities in information system development [Xu et al. 2012].

58
choosing the highest resolution model would be enough. However, the choice is made more difficult when the price, camera resolution, screen size, internal storage, and battery life all have to be taken into consideration.
Such problems frequently occur in modern information systems development as the number of requirements is often high. The requirements prioritization step solves conflicts among project stakeholders. The most important requirement for a customer is usually the cost to implement immediately. Additionally, software is developed incrementally, and prioritization techniques follow such increments [Ruhe 2005, Pegoraro and Paula 2017, dos Santos et al. 2016, Riegel and Doerr 2015.
Some techniques used to rank the requirements and to choose the most appropriate CA for an information system include the cost-value approach (CVA) [Karlsson and Ryan 1997], the analytic hierarchy process (AHP), numerical assignment, the 100-dollar test, the top-ten requirements, binary search tree, and bubble sort [van Lamsweerde 2009, Karlsson et al. 1998]. Among these techniques, AHP assists in complex decision-making, and it has been developed to analyze variables that are difficult to compare [Saaty 1980]. While AHP provides a hierarchical classification of requirements (e.g., ranking the cost in descending order), CVA implements a two-dimension approach to display the requirements' value against their cost and value [Karlsson and Ryan 1997]. The CVA process consists of applying the AHP method twice: one from a cost perspective and the other from the value perspective. In this paper, we state that 'cost' is the effort required to fulfill a requirement, and 'value' is the relevance of a requirement for a project [Karlsson and Ryan 1997]. In other words, 'cost' represents the effort exerted by the developer in working time to develop a given requirement, and 'value' represents the ease offered by a CA to fulfill a given requirement.
Given a software development project, our goals are: (i) to determine which nonfunctional requirements should receive more (or less) priority in each CA based on our slightly modified CVA method (SCVA method), and (ii) to determine which CA better fulfills a given set of requirements based on our method for choosing the most appropriate CA (MMACA method). The most appropriate CA is determined by our MMACA method based on a set of non-functional requirements. While several studies about CA focus from the perspective of management [Kazman et al. 1996, Christensen 2009], our focus is from the developer perspective.
Our approach compares desktop, web, and cloud architectures (i) to list the essential requirements and (ii) to determine the most appropriate CA given a set of requirements. We choose these three architectures to compare consolidated CAs (desktop and web) with a new CA (cloud). Briefly, a desktop CA runs software locally on a computer device; a web CA runs software over the Internet from a remote server as though local software; a cloud CA runs software as a service with rapid allocation/release and minimal effort from the service provider [Beaty et al. 2009, Voda 2014, Mell and Grance 2011.
Our SCVA method is applied to determine which requirement is better fulfilled by a CA, regardless of the project. Given a set of requirements as input, our method determines which requirements are relevant for desktop, web, and cloud CAs. Our SCVA method applies the CVA method, which, in turn, applies the AHP method twice (for cost 59 and value). We propose a generic set of requirements and therefore our method groups these requirements into two ways: stacked column charts and cost-value diagrams. The first analysis compares the aspects (cost and value) individually for each CA, and the second analysis combines both aspects to describe each requirement's priority based on both cost and value aspects. We call our method slightly modified because we apply the CVA method by development stages (planning, development, test, and operation), and we present the results into two ways (stacked column charts and cost-value diagram). In contrast, the CVA method [Karlsson and Ryan 1997] prioritizes requirements without distinguishing the development stage, and it presents a single analysis: a cost-value diagram.
Our MMACA method determines the most appropriate CA that best fulfills the information system requirements. Given a set of requirements as input, the method determines which CA better fulfills the input. Our MMACA method applies the SCVA method, calculates the ratio between cost and value for input requirements, and estimates the most appropriate CA (desktop, web, or cloud). The MMACA method considers the generic set of requirements built by the SCVA method.
Both methods require an initial set of requirements as input. Since there is no consensus in the literature, we built a set of requirements to evaluate our methods. We generated a set of 30 requirements based on bibliographical research, according to [Kitchenham and Charters 2007]. As our study focuses on the developer's viewpoint, our initial set of requirements was validated by a group of developers, generating a set of 25 requirements. This set is the input of the SCVA method, and any subset of this set is the input for the MMACA method. This subset may vary because it represents the most important requirements for an information system.
We performed an exploratory study to verify the feasibility of SCVA and MMACA. For the SCVA method, the developers evaluated the set requirements twice (once for cost and the other for value) and the SCVA method hierarchized the requirements by each CA.
For the MMACA method, we performed two evaluations. In the first evaluation, ten developers chose a subset of the requirements, and the method indicated the most appropriate CA. In the second evaluation, four developers from a real project created a single subset of the set of requirements, and the method indicated the most appropriate CA for this real information system.
Our results show the effectiveness and efficiency of our methods. Both SCVA and MMACA make potential contributions to the state of the art since they present novel methods for prioritizing requirements and choosing a CA.
The remaining of this paper is organized as follows: Section 2 presents an overview of CAs and requirements prioritization; Section 3 describes our related works; Section 4 presents our set of requirements and our methods (SCVA and MMACA); Section 5 describes validation our approach; Section 6 discusses our results; Section 7 presents some threats to validity; and finally, Section 8 presents our conclusions and future works.

Background
This section presents some background research on the topics supporting this work, starting with the definitions and details comparing the three CAs. Next, we examine requirements prioritization, focusing on the two techniques used: Analytic Hierarchy Process (AHP) and Cost-Value Approach (CVA).

Computational Architectures (CA)
Computational architectures (CA) are ways of structuring and organizing physical and logical devices. Architectures may differ in terms of memory management, data storage, processing capacity, and infrastructure [Patterson and Hennessy 2013].
In recent decades, application model sharing has evolved. Also known as a multiuser model, desktop architecture is user-focused because it hosts applications on a particular computer for each user. In this architecture, applications are limited by local hardware. Since applications are installed locally, the addition of non-functional requirements may require upgrading local hardware [Beaty et al. 2009, Wang et al. 2009. Although not recent, the choice for desktop architecture is justified for some reasons. First, the cost varies depending on the CA and requirements so there is no guarantee that applications for one particular architecture will always be the cheapest [Khajeh-Hosseini et al. 2010]. Second, companies with desktop applications would incur a high cost to migrate their applications to another architecture. For this reason, legacy systems collaborate with the use of this architecture [Khajeh-Hosseini et al. 2010, Maenhaut et al. 2016]. Third, migrating desktop applications to another architecture can cause organizational changes and make small business operations impossible [Khajeh-Hosseini et al. 2010, Armbrust et al. 2009]. Fourth, some web and cloud computing challenges (e.g., security, elasticity, and privacy) can be resolved on desktop applications more quickly [Khajeh-Hosseini et al. 2010, Armbrust et al. 2009].
With the Internet and the need to increase availability, web architecture has emerged which focuses on applications. The model is no longer multi-user (one computer for several users), but it becomes multi-application (one user starts to use several applications) in a client-server framework. The user accesses the application through client software (e.g., a browser) and part of the required storage and processing is made available by the application server [Wang et al. 2009, Voda 2014].
More recently, cloud computing has enabled on-demand access to a set of resources (e.g., networks, servers, storage, applications, and services), with rapid allocation/release, and with minimal effort from the service provider [Mell and Grance 2011]. This model can be understood as an advance of the web model [Erdogmus 2009, Pallis 2010]. The main features of cloud computing are: (i) the user can access new resources according to their needs; (ii) resources can be accessed from any device; (iii) similar resources are grouped; (iv) resources can be allocated and released automatically; and (v) payment is based on usage [Mell and Grance 2011].
Each CA implements a set of requirements to provide a structure to host applications. For example, while desktop architecture provides requirements for access to client resources and integration with local hardware, web architecture provides portability and 61 hardware consumption. On the other hand, cloud computing architecture provides availability and scalability. Because the requirements vary according to each CA, hierarchizing requirements considering multiple architectures is required.

Requirements Prioritization
Requirements can be prioritized according to one or more aspects. An aspect here is a project property used as a focus for comparison (e.g., value, cost, time, and risk). Coming back to the smartphone scenario, it would be like choosing a phone based on the best camera resolution. However, considering the resolution of the camera and the size of the screen, the resource that requires the most development effort (cost) may not be the most important (value).
Briefly, a requirement prioritization technique ranks a set's requirements according to some criteria. This prioritization can be done on the (i) ordinal, (ii) ratio, or (iii) absolute scale. Considering the cost aspect and the ordinal scale, requirements are analyzed one by one, and they are classified by cost. In addition to classifying, the ratio scale measures how expensive one requirement is compared to another (scale generally in percentage). Finally, the absolute scale hierarchy is performed based on an absolute value (number of hours to implement each requirement, for example) [Achimugu et al. 2014]. A complete list of techniques is available in [Achimugu et al. 2014]. Some prioritization techniques are described as follows.
Numerical Assignment is a technique that classifies the requirements into different groups (usually three or five). Each group represents a level in the analyzed scale, e.g., "critical, standard, and optional". A disadvantage of this technique is that prioritization considers only the group. Thus, requirements in a group have the same priority [Garg and Singhal 2017].
In the 100-dollar Test technique, each participant receives a certain amount of imaginary points (usually 100, considered as money or time) to allocate to the requirements. The result of each requirement will be between zero (0% important) and one (100% important). The problem of ranking many requirements can be mitigated with several points multiples of 100 [Santos et al. 2016, Ahuja et al. 2016].
In Top-ten requirements technique, given the set of all requirements, each stakeholder chooses their top ten, without assigning an order of priority to the requirements chosen. This technique is appropriate when there are several stakeholders involved, and they have equal importance [Achimugu et al. 2014].
The Binary Search Tree technique employs the structure of a binary tree and consists of the following: (i) initially, one of the requirements is positioned as the root node; then, (ii) each of the rest of the requirements is allocated in a tree, comparing each new requirement with the requirements already presented in the tree; finally, (iii) a tree with prioritized and balanced requirements is obtained [Bhandari and Sehgal 2014].
Bubble Sort organizes the requirements in a hierarchical array, and they are compared two-by-two. Thus the highest requirement (the most relevant, for example) is taken to the top (or highest position) of the array [Bhandari and Sehgal 2014].

62
The following subsections explain in detail both the requirements prioritization techniques used in our approach: AHP and CVA.

Analytic Hierarchy Process
AHP process can be explained in steps [Karlsson et al. 1998, Garg and Singhal 2017, Bhandari and Sehgal 2014, Dorado et al. 2014, Chung et al. 2006, Agarwal et al. 2013, Chaudhary et al. 2016, Saaty 1987]: 1. define the goal, variables, and comparison aspect 2. set up n requirements in an n x n matrix (each requirement will be one row and one column) 3. compare all requirements two-by-two according to a quantitative specific scale (presented in Table 1, detailed by [Saaty 1987]) 4. calculate the eigenvalues of the matrix with the average of the columns based on the calculated eigenvalues, each requirement receives a criterion value, and 5. finally, the consistency rate (CR) is calculated and, if inconsistent (this is, if CR > 10% [Saaty 1987]), then repeat steps 3 to 5.

Table 1. Quantitative specific scale for comparison of two requirements (i and j) in AHP process
Comparison value Interpretation 1 i and j are of equal value 3 i has a moderately higher value than j 5 i has a strongly higher value than j 7 i has a very strongly higher value than j 9 i has an extremely higher value than j 2, 4, 6, and 8 Intermediates and optional Concerning this kind of two-by-two comparisons (step 3), we can observe two important issues. First, if a ratio r between requirement i and j has value x (that is, if r(i, j) = x), then the same ratio r between j and i has value 1 x (that is, then r(j, i) = 1 x ). Second, the ratio r between a requirement i with itself always has value 1 (that is, r(i, i) = 1).
Step 6 aims to eliminate inconsistencies in evaluations. For instance, if r(i, j) = 3 (the ratio r between i and j is 3) and r(j, k) = 5 (the ratio r between j and k is 5), it makes no sense to describe r(i, k) = 1 3 (the ratio r between i and k is 1 3 ). For this, the consistency index (CI) is given by equation: where λ M AX is the largest eigenvalue of the judgment matrix and n is the number of evaluated criteria (requirements). Lastly, CR is given by equation: where CI is the consistency index (Eq. 1) and RI is the ratio index. The value of RI is fixed for each value of n (presented in Table 2, detailed by [Saaty 1987, Agarwal et al. 2013, Karlsson and Ryan 1997). Thus, the evaluation is consistent if CR ≤ 10% [Saaty 1987].

Cost-Value Approach
The AHP method must be applied twice to gather the CVA method, considering both value and cost criteria. Then, a two-dimension graph is generated with the cost on the x-axis and the value on the y-axis, both as percentages [Karlsson and Ryan 1997].
Once the graph is plotted, the requirements are classified with high, medium, or low priority based on each requirement's value and cost. The graph is divided equally into three distinct areas: (i) requirements with a high priority, i.e., high ratio between value and cost (a value/cost ratio greater than 2); (ii) requirements with a medium priority, i.e., medium ratio between value and cost (a value/cost ratio between 0.5 and 2); and (iii) requirements with a low priority, i.e., with a low ratio between value and cost (a value/cost ratio lower 0.5) [Karlsson and Ryan 1997].
The strengths of the CVA technique are: (i) it considers two aspects (value and cost), (ii) it groups the requirements by priority levels, (iii) it has a systematized process, and (iv) it allows the comparison of criteria of different categories.
Among the alternatives for requirements prioritization, to the best of our knowledge, CVA is a good option for our problem since the technique relates two distinct aspects in a single analysis, and it groups requirements by three priority levels.

Related Works
This section presents related research regarding our problem. We can divide these into two categories: (i) AHP and CVA use experiences and (ii) a CA selection. We describe some related works on both topics, emphasizing the similarities to and differences from our approach.
In addition to the works discussed in the following sections, there are some other related works in the literature.
Cost-Benefit Analysis Method (CBAM) [Kazman et al. 2002] and Economics-driven Software Architecture [Thurimella and Padmaja 2014] are examples of approaches that analyze the economic impact of architecture decisions. Value-based Software Engineering (VBSE) [Boehm and Huang 2013] and VALUE framework [Mendes et al. 2018] address a broader view of value in software-related decisions. Unlike our approach, all four works consider one aspect only (cost or value). Value-based approaches (e.g., VBSE and VALUE framework) consider value as personal attitudes and beliefs that may influence 64 requirements. In this paper, we consider value as the effort required to code a given requirement.

Both AHP and CVA use experiences
Dorado et al. [Dorado et al. 2014] proposed a method to choose software for engineering simulations, considering educational criteria such as interface, learning effectiveness, portability, help support, and cost. Chiam et al. [Chiam et al. 2013] developed an approach for selecting quality attributes in software. The authors present a method that considers risk management and process integration with the AHP method. An industry case study is performed to validate their method. Unlike our approach, authors in [Dorado et al. 2014, Agarwal et al. 2013, Chaudhary et al. 2016, Chiam et al. 2013] apply the AHP method to a single value not relating two criteria: value and cost.
CVA was originally proposed by Karlsson and Ryan [Karlsson and Ryan 1997], and they prioritized the requirements of two case studies: the radio access network project (with 14 generic requirements), and the performance management traffic recording project (with 11 generic requirements). The goal of the first case study was to identify and specify requirements for a mobile system. The second case study is a software system for recording and analyzing traffic. Chung, Hough, and Ojoko-Adams [Chung et al. 2006] prioritized software security requirements using the CVA method. Authors in [Karlsson andRyan 1997, Chung et al. 2006] apply CVA to relate the value and cost of requirements in particular projects. However, they do not evaluate their results to compare different CA. Although the original purpose of the CVA method is to compare requirements [Karlsson and Ryan 1997], to the best of our knowledge, this is the first work to apply CVA to compare and select CAs.
The work most similar to ours is summarized by [Galster et al. 2010]. Galster, Eberlein, and Moussavi [Galster et al. 2010] propose a method for systematic selection of architecture styles (called SYSAS). SYSAS selects an architectural style (e.g., Pipe-andfilter, Client-server) based on desired properties of the system and properties of architectural elements (e.g., persistence, migratability, synchronicity, efficiency, scalability). This work compares the results of SYSAS with case studies judged by experts. Unlike our approach, which applies the CVA method with two aspects (cost and value), the authors in [Galster et al. 2010] conducted their study on a single aspect of importance (through the AHP method). Moreover, our work ranks the set of requirements by computational architecture.

Selection of CA
Christensen [Christensen 2009] proposes an architecture for future mobile applications based on cloud computing, web services, smart mobile devices, and context. In their study, smart mobile devices use sensors to get the application context, making applications smarter. These applications can be enhanced with the advantages of cloud computing to exceed the traditional capabilities of mobile devices. In the same context, Huang, Xing, and Wu [Huang et al. 2013] argue that mobile devices have become one of the major service providers. The authors propose a new model of mobile cloud computing with the focus on the user. In this approach, devices, clouds, and servers are highly interactive.
In contrast to our paper, the authors in [Christensen 2009] propose improvements to a particular architecture.
The software architecture analysis method (SAAM) is a method proposed by Kazman, Abowd, and Clements [Kazman et al. 1996] that facilitates the choice of CA based on scenarios, brief narratives of expected or anticipated system uses. SAAM creates scenarios (brief narratives about system uses) to provide a view of how the analyzed system satisfies quality attributes in various usage contexts. The method is composed of five activities: (i) description of the candidate architecture, (ii) creation of scenarios, (iii) evaluation of each scenario (in isolation), (iv) evaluation of the interaction between the scenarios, and (v) determining the weight of each scenario and its interactions. These activities follow a defined flow, and it can be applied for early problem detection and enable a better understanding of the architecture. For example, in the case study presented in the paper, the authors found limitations in obtaining specific requirements such as portability. Although this work lists conflicts between requirements and architectures, the proposed method is manual, and it does not classify requirements according to priority.

Our SCVA and MMACA methods for CA
The problem of choosing a CA is directly proportional to the number of requirements. This means that increasing the number of requirements increases the complexity of making a decision, and increases the complexity of determining the requirements with high priority.
Based on a literature search for non-functional requirements, our approach presents a set of requirements that guide the developers' choice of a CA for a given application. This approach comprises of (i) a set of requirements, grouped into four software development stages, (ii) a slightly modified CVA method (SCVA) to analyze priority requirements by CA, and (iii) a method for choosing the most appropriate CA (MMACA) based on a subset of the requirements. Examples of users of the SCVA method are software architects and senior developers, while examples of users of the MMACA method are other developers. Fig. 1 summarizes the design of our approach. We emphasize that SCVA (step 2) has a set of requirements as input (step 1) and three subsets of this set of requirements (subsets with the highest priority requirements in desktop, web, and cloud CA: one subset for each CA) as output. The MMACA (step 4) has a set of requirements such as input (step 3) and CAs most appropriate for these requirements (in order of suggestion: first, second, and third) as output.
Step 3 groups the requirements into low, medium, and high priority by CA. This step is the SCVA method output and MMACA method input.
In this section, we present the set of requirements considered, and both the SCVA and MMACA methods in detail.

The Set of Requirements
This section details the set of requirements necessary to perform both SCVA and MMASA methods. Initially, we present the adopted strategy to search and select requirements. Next, we evaluate the set of requirements. Finally, we present the final set of requirements and the methodology overview.
-high priority -medium priority -low priority Figure 1. Design of our approach: SCVA has a set of requirements as input and the highest priority requirements by CA as outputs; MMACA has a set of requirements such as input and CAs most appropriate for these requirements as output.

Requirement search
The set of requirements was constructed based on the guideline proposed by [Kitchenham and Charters 2007]. We conducted a literature search for papers that present non-functional requirements. This literature search was conducted by two researchers with moderate experience in requirements engineering. The search method was as follows: 1. a search string was constructed to cover the largest amount of related works. After tuning, the field string was "software engineering" AND (requirements OR features) AND architecture AND (desktop OR web OR cloud) 2. the string was applied in four databases: Science Direct, IEEE Xplore, ACM Digital Library, and Springer 3. finally, papers were evaluated, and three exclusion criteria were applied: (i) papers not written in English, (ii) papers not published in a journal, a conference, or a workshop, and (iii) papers unrelated to CA, computational architecture, or development.
The final set of papers was [Kazman et al. 1996, Garg and Singhal 2017, Chieu et al. 2009, Cunsolo et al. 2009, Chien et al. 2003, Rewatkar and Lanjewar 2010. Appendix A presents the titles, authors, and references of the papers. For each paper, we identified the non-functional requirements, a total of 30 items. We identified the requirements from the literature with the help of two developers. Both developers have a master's degree and more than 5 years of experience in the field.

Requirement evaluate and filter
Six developers evaluated the set of 30 requirements. The participating developers were asked to answer the characterization questionnaire regarding the professional experience time (in years), self-evaluation (in Likert scale), and experience in requirements engineering (in Likert scale). Four developers have considerable experience (in Likert scale) with requirements engineering and two developers have moderate experience (in Likert scale).  Each developer reported whether or not each requirement was significant for a comparative study among CAs. Thus, each requirement received a binary evaluation y-n, indicating that y developers answered 'yes' (yes, the requirement is relevant to the study), and n developers answered 'no' (no, the requirement is not relevant to the study).
Given each developer's opinion, we applied the Kappa Coefficient [Fleiss et al. 1971] to determine the level of agreement between responses and then to determine a set of relevant requirements.
Considering the analysis made by [Cyr and Francis 1992], our Kappa Coefficient for the set of 30 requirements according to the developers' opinions was k = 0.784. Results between 0.61 and 0.80 mean substantial agreement [Cyr and Francis 1992].

The final set of requirements
With 25 requirements and high agreement, we generated a single set with the most significant requirements. Although y-n evaluation was carried out regardless of CA, each requirement has advantages and disadvantages depending on the architecture. For instance, portability may have a higher cost in desktop rather than web architecture, or the scalability test can be more appropriate in a cloud architecture.  The set of requirements was organized into four developments stages according to lifecycle [van Lamsweerde 2009]: (i) planning consists of the first two steps (requirements analysis and definition, and software design) and it contains the requirements that must be considered before coding; (ii) development is the stage of implementation (codification) of a software; (iii) test is responsible for verifying the correctness of a design requirement; and (iv) operation involves the requirements derived from the use and maintenance of a system (Table 4). For each requirement in the set of requirements, we summarize the development stage and binary evaluation y-n.
Both SCVA and MMACA methods employ the stable artifact present in Table 4. While the SCVA method groups the requirements by CA and priority, the MMACA method lets the stakeholders select the set of the most important requirements for their projects and then the MMACA method determines the set of the most appropriate CA for those requirements. A description of each requirement is available at https://github.com/FORMAS/detFSR. Requirements are grouped into four development steps to reduce the number of comparisons.
Since n requirements need (n * (n−1)) 2 comparisons [Karlsson and Ryan 1997], 25 requirements need 300 comparisons. The division by stages of development allows comparisons by each group (Table 4 shows the number of requirements for each stage of development). Thus, the comparisons are reduced from 300 to 70: 21 for planning, 28 for development, 6 for testing, and 15 for the operation stage.
Furthermore, the division emphasizes the development stage of requirements with higher priority and allows the methods to focus on the current stage of the application. For instance, developers of a given application may need to know the requirements with high priority for the operation and test stages, but not the planning stage. On the other hand, the operation stage may not be important for an application until the planning stage.
After that, we obtain the set of requirements: a set of 25 requirements, grouped by development stage, and with a high degree of agreement among the developers interviewed. The set of requirements is applied in different ways depending on the proposed method.

Overview
Fig. 2 summarizes the design applied to obtain the set of requirements. Initially, we conducted a literature review of papers about non-functional requirements for projects in desktop, web or cloud architectures (step 1). We selected eight papers, and we extracted 30 different non-functional requirements (step 2). Next, we invited six developers to evaluate the set of 30 requirements (step 3) and the Kappa Coefficient result was k = 0.784 (step 4). Results between 0.61 and 0.80 mean substantial agreement [Cyr and Francis 1992]. Since almost perfect agreement requires values greater than 0.81, we performed a new round. In the new round, five requirements evaluated as 4-2 and 3-3 were removed (step 5), and the new Kappa Coefficient result of k = 0.853 (step 6). These steps resulted in a set of 25 requirements (step 7). The developers participated only in the process of judging the requirements (step 3).

A Slightly Modified CVA (SCVA) for CA selection
Our Slightly Modified CVA (SCVA) for CA selection is introduced in two parts: description and SCVA use example.

Description of SCVA
SCVA determines which requirements have the highest priority in desktop, web, and cloud architectures. For this: (i) we separate the set of requirements according to the development stage, (ii) we apply the CVA method for each development stage, and finally, (iii) we list the highest priority requirement for each CA. The SCVA method must run once for each CA. Fig. 3 illustrates the design of our SCVA method: CVA is applied individually for each development stage (CVA 1 , CVA 2 , CVA 3 , and CVA 4 ), and individual results are presented in two analyses (R 1 and R 2 ). The first analysis (R 1 ) compares the aspects individually for each CA, i.e., given one requirement and one aspect, we can emphasize in which CA this requirement is more (or less) costly (or valuable). On the other hand, in R 2 the requirements are grouped by CA, i.e., given one requirement and one CA, we can emphasize the priority of this requirement (high, medium, or low)   While our SCVA applies a particular set of requirements in this work, it can be performed with any other set of requirements. For this, it is only necessary to follow the steps described in the next section.

SCVA use example
Considering that SCVA applies CVA, which in turn applies AHP twice, we show part of the requirements hierarchy process below. The example considers requirements of the operation stage (Table 4) of a project in cloud computing architecture. In this section we present the analysis for the cost aspect, and then, we perform the analysis considering the value aspect, as required by the CVA method. Table 5 presents the n × n matrix after two-by-two comparison by developers for the cost aspect, operation stage, and cloud architecture. Requirements are ranked according to the AHP method. Table 6 presents the classification for the comparison in Table 5. The "rank" column indicates the degree of the requirements, and the "order" column indicates the position of the requirements in the set. In this case, the order is inversely proportional to the cost. For instance, Control of unexpected events is the most costly requirement, while Backup is the cheapest (considering the CA cloud). As the consistency ratio (CR) in this example is 5.2% ≤ 10%, the evaluation is consistent.
Steps (including the calculation of CR) were performed using the online tool BPMSG AHP Online System 1 and Excel worksheets.

Method for Choosing the Most Appropriate CA (MMACA)
Our Method for Choosing the Most Appropriate CA (MMACA) is introduced in two parts: description and the MMACA use example.

Description of MMACA
Choosing the most appropriate CA is a task that involves many factors because the best CA depends on the business model and application features [Patterson andHennessy 2013, Bessa et al. 2016]. Generally, this choice is made informally according to business rules. The choice based on a set of requirements may be complex or at least leaves room for discussion. Therefore, we present a method to assist in choosing the most appropriate CA for a given set of requirements. The MMACA method applies w requirements of the set of requirements present (Table 4), where 1 ≤ w ≤ 25.
Among the requirements in the set (Table 4), let us suppose that stakeholders determine which requirements are the most important for a project, creating a subset with w requirements. Based on the values of our comparisons, this subset can be used to suggest the most appropriate CA (an example is described in Table 7). For this analysis, our method for choosing the most appropriate CA (MMACA) follows these steps: 1. among the requirements in the set (Table 4), stakeholders determine the most important requirements, creating one subset with w requirements 2. for each requirement in the subset, the ratio between value (in %) and cost (in %) is calculated considering CAs individually (according to [Karlsson and Ryan 1997]) 3. after estimating value divided by cost, a geometric mean of ratios for each CA is calculated 4. finally, CA with the highest geometric mean is the most appropriate.
Stakeholders may have different opinions about the same project. This means that given the same project, if two stakeholders apply the MMACA method, both can use different inputs.

MMACA use example
Since requirements value and cost are available for different CAs, the decision of which one to use can be taken more securely. For instance, assume that requirements with ids 2, 9, 14, 21, and 22 (according to Table 4) are important for a project. In this case, the next step is to calculate the relationship between value and cost of each important requirement for each of the three CAs. Finally, we must calculate the geometric mean for each CA. Thus, the mean of the highest value represents the most appropriate CA.
Based on the method presented and the data presented in Table 7, the best choice for input above (2, 9, 14, 21, and 22) is web architecture, and cloud architecture as the second option. In contrast, assuming input with 4, 6, 8, 12, 16, 18, 21, and 24, the best choice is cloud architecture, and desktop the second choice. iSys: Revista Brasileira de Sistemas de Informação (iSys: Brazilian Journal of Information Systems) https://sol.sbc.org.br/journals/index.php/isys Table 7. Example of architecture choice based on an input with ids 2, 9, 14, 21, and 22. Each row (except the last one) represents the ratio between value and cost for a particular requirement. Columns group the ratios by each CA. The last row presents the geometric mean of the ratios for each CA Both methods can be applied jointly or independently: stakeholders may apply SCVA only, MMACA only, or both SCVA and MMACA, depending on the current stage of the project. For instance, the SCVA method may determine the most priority requirements of projects that are in the early planning stage. On the other hand, the MMACA method may determine the most appropriate CA of projects that are in the final coding stage (or implementation stage). When it is necessary to apply both SCVA and MMACA, one possibility is to consider the output of the SCVA method as the input of the MMACA method.

Validation of SCVA and MMACA approaches
We performed the SCVA method with the set of requirements (Table 4) and MMACA method with some inputs to evaluate our approaches. To illustrate the SCVA and MMACA methods, (i) we analyze the requirements and classify the requirements in low, medium, and high priority, and (ii) we classify the most appropriate CA for ten generic projects and one real project (middleware for DaaS and SaaS [Ribeiro et al. 2019]).
Before the two validation rounds, we collected professional data about desktop, web, and cloud architectures. Initial steps were (i) the construction of the design of the experiment and (ii) filling a form. Design is an essential artifact in software engineering experiments, and the form aims to collect developers' professional experience in years and self-evaluation.
Stakeholders must follow the sequence of activities proposed by our methods. These activities generate input and output artifacts during their execution. In order to evaluate the effectiveness and efficiency of our methods, we are interested in two research questions: RQ1. Can developers apply our methods and get the outputs? This research question explores the effectiveness of our approach by assessing whether developers can apply our methods according to their expertise. RQ2. Do the developers agree with the outputs? This research question explores the efficiency of our approach by assessing whether developers agree with the results 74 obtained by our approach. In other words, we are interested in whether our results match the requirements of the evaluated projects.
Both RQ1 and RQ2 are important because our method would be unfeasible if stakeholders could not carry out the activities and generate the artifacts correctly. Therefore, RQ1 and RQ2 are related because RQ1 assesses the feasibility of carrying out the method and RQ2 assesses whether the results agree. Fig. 4 illustrates the SCVA method validation. After two initial steps, the 30 requirements initially collected were judged and evaluated by six developers. They judged the relevance of each requirement generally, i.e., without considering a particular project. We created a unique set with the most important requirements to apply AHP to the SCVA method with the answers.  After applying the SCVA method, the requirements grouped by aspect reveal the CA in which the requirements have more (or less) value or cost. Stacked column charts provide a macro view of data. In our design, shown in Fig. 3, we call this analysis of R 1 . Fig. 5 describes the requirements considering the value aspect, and Fig. 6 illustrates the cost view. For example, according to Fig. 5, requirement 16 is more relevant for desktop architecture than for cloud architecture, and requirement 19 is more relevant for web architecture than desktop architecture. In contrast, as Fig. 6 shows, requirement 1 has a higher cost in desktop than web architecture, and requirement 25 has a higher cost in cloud architecture than desktop architecture. Fig. 7, 8, and 9 present the cost-value diagram for each CA, showing the requirements with the highest priorities. Both diagrams are useful for the composition of the requirements as they are grouped according to the ratio between value and cost values. We present this analysis in our design of Fig. 3 Table 4).

SCVA method validation
Some open issues are important to discuss: (i) requirement 13 has high priority in both CAs, (ii) Requirements 5, 13, and 22 have high priority in desktop and web architectures, as well as 13 and 17 has high priority in web and cloud architectures, (iii) requirements 3, 4, 6, 7, 8, 9, 11, 12, 16, 19, 21, 23, 24, and 25 are not present in the set of high priority for no CA, (iv) only desktop architecture does not contain requirements of the test step as a high priority, and (v) all CAs have at least one planning, development, and operation requirement which is high priority.

77
based on their expertise. Given that evaluating the expertise is subjective, we assume that the developers' opinion for the SCVA method is optimal.
The output of the SCVA method is the input of the MMACA method. Although running the SCVA method is laborious (i.e., AHP eight times), this method needs to be performed only once for each set of requirements. On the other hand, the MMACA method can be performed several times for each SCVA method's output. We emphasize that the benefits outweigh the cost of applying our approach.

Generic Projects
These steps enabled the evaluation of RQ1 and RQ2. The developers applied our MMACA method to rank CA that best fulfills a set with the most relevant requirements, and 90% of the developers agreed with the results. By doing this, we evaluated the effectiveness and efficiency of our method.

Real Project
Middleware for DaaS and SaaS (MIDAS) provides interoperability between cloud services such as Software as a Service (SaaS), Data as a Service (DaaS), and Database as a Service (DBaaS) [Ribeiro et al. 2019]. MIDAS mediates communication between SaaS and DaaS/DBaaS transparently. The current version of MIDAS is hosted on Heroku (https://www.heroku.com), an open cloud that provides enough storage space and a complete platform for the project. We select MIDAS to evaluate our MMASA method since we have access to the middleware source code and developers, and this allows us to evaluate features using an existing framework. Fig. 11 illustrates the use example of the MMACA method in the middleware MIDAS.
In step (1), each of the four developers pointed out the requirements of set ( Table  4) that MIDAS should implement. At the end of this first step, each requirement was associated with an evaluation y-n, indicating how many developers voted 'yes' (yes, MIDAS must implement such a requirement) and 'no' (no, MIDAS should not implement such a requirement). For instance, the requirement availability (id 6) was evaluated 3-1 (3 votes 'yes' and 1 vote 'no'), while the requirement security (id 1) was evaluated 1-3 (1 vote 'yes' and 3 votes 'no'). Table 9 displays the compiled evaluations of the four developers for the current 25 requirements of the set of requirements.
In step (3), to increase the level of agreement of the collected opinions, the evaluations without an absolute majority (that is, evaluations of type 2-2: 2 votes 'yes' and 2 votes 'no') were removed. The requirements Portability, Resource elasticity, End user test, Device sync test, and Control of unexpected events (ids 8, 12, 17, 19, and 25, respectively) were removed, and the level of agreement was again evaluated as strong (k = 0.65) in step (4).

80
Finally, the MMACA method was performed based on 14 requirements important for MIDAS and the cloud was suggested as the most appropriate CA for the middleware, and web as the second option.
Since MIDAS is hosted in the Heroku cloud, this evaluation enabled the evaluation of RQ1 and RQ2. Developers applied our MMACA method to rank CA that best fulfills a real project (middleware MIDAS). Our result indicated cloud computing architecture as the first option for MIDAS, exactly the current CA. In this way, we evaluated the effectiveness and efficiency of our method.

Discussion
Information systems are increasingly complex, and this complexity is also directly proportional to the number of stakeholders and functional and non-functional requirements that should be fulfilled in a software project. While some requirements are codified into the development stage, others will be provided by CA. Choosing the most appropriate CA for a set of requirements is still a challenge because different architectures offer a distinct set of requirements and ways for this fulfillment. This choice is usually direct assigned to developers, not end-users.
Both SCVA and MMACA methods aim to systematize RE activities to choose the most appropriate CA for a set of requirements. While our SCVA method supports developers based on the literature (i.e., domain's stable knowledge), our MMACA method focuses on application-specific demands. Software engineers can also use our methods to evolve its artifacts by adding new requirements and adapt to requirements previously analyzed.
Our proposal intends to group consolidated (i.e., literature) and stakeholders' specific knowledge to point out the most appropriate CA. Several works aim to present and discuss different CA requirements fulfillment. Our proposal is related to the existing literature in two ways. First, requirements listed by our related works (e.g., [Kazman et al. 1996]) can compose our SCVA method. For this purpose, new requirements need to be compared with previous others (as shown in Fig. 3) from the three architectures' perspective, and so generating a new set of requirements, new stacked column charts (R 1 in Fig. 3), and new cost-value diagrams (R 2 in Fig. 3). Second, CA listed by other works (e.g., [Christensen 2009] and [Huang et al. 2013]) can also be part of our approach. In that case, new CAs need to be included in the requirements comparison activity (CVA method step in Fig. 3), generating new new stacked column charts (R 1 in Fig. 3) and new cost-value diagrams (R 2 in Fig. 3). The new requirements or CAs addition implies the need to perform our methods in an incremental approach since the MMACA method is performed based on the SCVA method results.
The approach generalization is related to replication our and new studies as the generalization of one architecture's knowledge depends on a generic set of requirements to cover a CA's characteristics in detail. We can gradually make the MMACA method more accurate by applying the SCVA method several times based on a more refined set of requirements. In summary, our methodology can be generalized by (i) expanding the set of requirements used in both methods (Table 4) and (ii) reproducing the methods 81 until stabilizing the knowledge about the applicability of a given CA for the same set of requirements. Our experiments are available at FORMAS Research Group website 2 and may be replicated in other studies. We hope that sharing this methodology version provides future contributions and improvements.

Threats of Validity
Our approach is subject to limitations. According to [Wohlin et al. 2012], threats to the validity of a study contribute to the trustworthiness of the results. In this section, we present the threats identified during this research.
Conclusion validity addresses the ability to obtain a correct conclusion from the relationship between the treatment and the outcome of the experiment [Wohlin et al. 2012]. In this paper, so that developers' experience did not influence the results, we consulted developers with varied and random experience, without any grouping. Furthermore, the number of developers interviewed could be considered low.
Internal validity addresses any factor that may affect the study's independent variables without the researchers' knowledge [Wohlin et al. 2012]. In this paper: (i) as different developers could interpret the same requirement differently, we give a brief description of each requirement, (ii) since there is an initial complexity to the AHP method, the comparisons were collected through interviews, (iii) in order to homogenize the selection of the developers, all the interviews were conducted on a voluntary basis, and (iv) in order for a developer's response not to influence another developer's response, the interviews were carried out individually.
Construct validity addresses factors (e.g., design or social) which make it difficult to generalize the results [Wohlin et al. 2012]. In this paper, to avoid social threats, developers were not evaluated based on performance. Additionally, the MMACA method assumes that stakeholders are aware of the non-functional requirements of their projects. External validity addresses conditions that limit the generalization of the results of our experiment to industrial practice [Wohlin et al. 2012]. In this paper, in order to facilitate generalization, only developers with some experience participated in the experiment.

Conclusion
This paper analyzes a set of requirements to compare desktop, web, and cloud architectures. Moreover, we investigate the effects of choosing the architecture that best fulfills a set of requirements. First, requirements were classified based on the AHP method. These requirements were then grouped into priority levels based on the CVA method. High priority requirements simplify the comparison of CAs. Similarly, the most appropriate CA analysis helps decisions where requirements are particularly important for an information system.
Based on the questions RQ1 and RQ2, our results suggest the effectiveness and efficiency of the approach. Our research found that the CVA method is useful for prioritizing requirements, choosing a CA according to an information system's requirements. As 82 contributions, we compare different CAs based on a set of non-functional requirements (according to the SCVA method), and we present a strategy to facilitate the choice of a CA based on the potential requirements of a project (according to the MMACA method). As expected, we emphasize that the most appropriate CA depends on the requirements, available resources, and stakeholders' priorities.
The major limitation of this study is that the CVA method does not consider the dependencies among requirements. For instance, a low priority requirement may be required for the implementation of a high priority requirement.
Although we are aware of the limitations, validation answered the research questions positively. We intend to refine our strategy to provide better results. Additionally, we are working on (i) adding fog computing architecture [Bonomi et al. 2012] to the analysis of a new paradigm that emerges as an extension of cloud computing, and (ii) applying our MMACA method in more projects to generalize our results.