Mining Experts from Source Code Analysis: An Empirical Evaluation

Modern software development increasingly depends on third­party libraries to boost productivity and quality. This development is complex and requires specialists with knowledge in several technologies, such as the nowadays libraries. Such complexity turns it extremely challenging to deliver quality software, given the pressure. For this purpose, it is necessary to identify and hire qualified developers, to obtain a good team, both in open source and proprietary systems. For these reasons, enterprise and open source projects try to build teams composed of highly skilled developers in specific libraries. However, their identification may not be trivial. Despite this fact, we still lack procedures to assess developers skills in widely popular libraries. In this paper, we first argue that source code activities can identify software developers’ hard skills, such as library expertise. We then evaluate a mining­ based strategy to reduce the search space to identify library experts. To achieve our goal, we selected the 9 most popular Java libraries and 6 libraries for microservices (i.e., 15 libraries in total). We assessed the skills of more than 1.5 million developers in these libraries by analyzing their commits in more than 17 K Java projects on GitHub. We evaluated the results by applying two surveys with 158 developers. First, with 137 library expert candidates, they observed 63% precision for popular Java libraries’ used strategy. Second, we observe a precision of at least 71% for 21 library experts in microservices. These low precision values suggest space for further improvements in the


Introduction
Software development has become increasingly complex, both in opensource and proprietary systems (Damasiotis et al., 2017). Such complexity makes it extremely challeng ing to deliver software with quality in time and may hinder developers' participation in worldwide repositories of source code, such as GitHub . To contribute to opensource projects or hire developers (in the case of a company), identifying the developer with the right skills for a good team is a hard task (Garcia et al., 2007; McCuller, 2012. Besides, in many cases, project managers must build teams of skilled developers in relevant libraries. However, decisions made during the hiring process are a wellknown decisive factor to the success of a software project (Tsui et al., 2016). Providing a more reliable way of identifying developers' skills can help project managers make the right decision when hiring or attracting the right developers for an opensource project. The task of finding experts in spe cific technologies is especially complex, despite the exis tence of businessoriented social networks, such as LinkedIn, where developers write about their attributes and qualifica tions. This type of platform is commonly used for the online recruitment of professionals. However, the reliability and ac curacy of the information provided in such media are not guaranteed (Brown and Vaughn, 2011). For instance, some individuals can overvalue their skills and omit some skills in a selfauthored curriculum.
The most commonly used strategies to find experts have their limitations (Tsui et al., 2016; Constantinou andKapit saki, 2016). For instance, the analysis of the curriculum from LinkedIn or in paper format can omit desirable skills. Be sides, developers may have difficulty to express their qual ifications (Tsui et al., 2016). Sometimes, the developer has a specific ability, but s/he considers it irrelevant. In another situation, the developer cites many skills but does not have expertise in the technologies mentioned (Constantinou and Kapitsaki, 2016). Even large companies may rely on cur riculum analysis, and this type of research may have inaccu rate or outdated information. Besides, even talent recruiters may incorrectly identify the developer skills or identify other skills that are not the organization's focus. Hiring lowly skilled software developers can lead to additional costs, ef forts, and resources for training them, or expending more time and resources hiring others (Constantinou andKapit saki, 2016; Sommerville, 2015). However, these costs can be reduced if companies identify with more precision best developers according to a job opening.
Several software developers have used social coding plat forms, such as GitHub and BitBucket, to showcase their work, hoping that this may help them be hired for a better job. Developers use these social coding platforms to demon strate their skills and create an online profile about their projects (Constantinou and Kapitsaki, 2016). Some contrib utors are even using these platforms' social aspects to in fer project popularity trends and promote themselves more efficiently through specific projects and collaborations in other opensource projects (?). In some cases, profiles de rived from accounts of social platforms, such as GitHub, are considered even more reliable than a curriculum from LinkedIn, concerning the technical qualifications of a job candidate (Constantinou and Kapitsaki, 2016). Therefore, data exploitation from coding platforms is a promising way for potential employers to identify and assess several candi dates in real situations (Capiluppi et al., 2013).
GitHub has been widely used in several works mainly be cause it provides several userbased summary statistics, such as the number of contributions in the last year, the number of forked projects, and the number of followers. For instance, some works have used this platform to identify appropriate developers for crossproject bugs (Ma et al., 2017), iden tification of reuse opportunities (Oliveira et al., 2016) and collaborations between projects (Dabbish et al., 2012). Dif ferent approaches have been used to investigate the skills of developers from GitHub (Saxena and Pedanekar, 2017; Mockus and Herbsleb, 2002; Greene and Fischer, 2016. For instance, prior work conducted interviews with members of GitHub to understand the hiring process (Marlow and Dab bish, 2013). We did not compare the results with other ap proaches because our strategy is very different from the oth ers. Therefore, our strategy complements related work by au tomatically reducing the search space to support library ex perts' identification. This paper is an extension of our previ ous work  that proposed and evaluated a strategy to identify library experts from source code, named JExpert. Our main goal is to reduce the search space to iden tify library experts. We list the following new contributions to this submission compared to the original paper.
1. We present and analyze data of all identified expert can didates by means of new boxplot charts. 2. We include a novel classification and discussion of ex perts in four categories. 3. We include additional analysis of the library experts by proposing a novel heuristic to rank the top experts of each library. 4. We perform a new identification of experts in microser vices libraries. 5. We conducted an additional survey to calculate the strat egy precision on identifying experts in microservices. 6. We include additional discussion about the negative re sults of the evaluated metrics.
In this paper, we evaluate the feasibility of identifying soft ware developers' hard skills; that is, library expertise from source code analysis. We rely on GitHub data to support the identification of the skills of developers based on their contributions. From each type of developer contribution, we aim to identify essential developers skills and evaluate the applicability and precision of the strategy. In the applicabil ity evaluation, we performed a mining study with the top9 most popular Java libraries from GitHub, aiming to identify library experts in these libraries. In total, we analyzed more than 16 thousand projects and 1.5 million developers. In the precision evaluation, we designed and sent a survey to more than 1 thousand developers identified for these libraries. We received answers from 158 developers. As a result, we ob serve that it is possible to reduce the search space to identify experts from source code. We also note that the strategy pro vides meaningful information to recruiters, such as the his tory of written lines of code (LOC) for each library. These details about the developers can improve the selection of can didates.
Our key contributions are threefold: • we empirically evaluate the applicability and precision of identifying library experts from source code analysis. In addition, we propose a tool to support the strategy; • we identify 1,045 experts in top9 Java libraries with a precision of about 63%; • we identify 136 experts from microservices libraries with a precision of about 71%.
Low precision values indicate space for future research in this subject. The remainder of this paper is organized as fol lows. In Section 2, we describe our analysis by detailing the strategy to identify library experts, dataset, and our research questions. Section 3 presents the results of the applicability evaluation to identify library experts. Section 4 shows the re sults to survey with top9 library experts. Section 5 shows the results concerning a survey with library experts in microser vices. Section 6 shows details about a tool developed to sup port the strategy. Section 7 presents and discusses threats to validity. Related work is discussed in Section 8. Finally, Sec tion 9 discusses the concluding remarks and future work.

Study Settings
This section describes the protocol to evaluate the identifica tion of library experts through an empirical study. Section 2.1 presents the aims of our study and the research questions we address. Section 2.2 shows the steps performed to evaluate the expert candidates. Section 2.3 describes the used dataset.

Goal and Research Questions
This study's primary goal is to evaluate the applicability and precision of a strategy to reduce the search space to identify li brary experts from source code analysis using software repos itories. We are interested in whether the strategy can signifi cantly reduce the search space to identify experts in a specific library. We are also concerned with assessing the relevance of the results provided by the strategy. For this purpose, we select the 10 most popular and standard Java libraries among GitHub developers. We also selected 6 popular libraries for microservices. One library was later excluded (Section 2.3). Therefore, we evaluate the strategy with the 9 most popu lar Java libraries and 6 libraries of microservices. To achieve this goal, we use the GoalQuestionMetric method to select measurements of source code. The GQM method proposes a topdown approach to defining measurement; goals lead to questions that are then answered with metrics (Basili et al., 1994). Table 1 shows the GQM with the research questions and metrics investigated in this study. As mentioned, the goal of this paper is to reduce the search space to identify library ex perts from source code. Therefore, from this goal, we check if it is feasible to analyze the source code to identify library experts. Through RQ1, we are interested in investigating the efficiency of the number of commits (metric) to indicate the level of activity of a developer in a specific library. In other words, we aim to analyze the number of commits involving a specific library performed by a developer to compute their activity level in the library.
With RQ2, we aim at assessing the knowledge extension based on the number of imports to a specific library. From all imports made by a developer at the source code, we in vestigate the number related to the particular library. Finally, the last research question (RQ3) analyzes the knowledge in tensity of the developers from the number of LOC related to the library (metric). In this last question, we aim to evalu ate the amount of LOC implemented by a developer using a specific library. For this purpose, we evaluate the relation of total LOC and LOC related to a particular library.

Evaluation Steps
This section describes the steps to evaluate the identification of library experts from source code. To answer the research questions presented in Section 2.1, we designed a mixed method study composed of four steps: 1) Library Selection, 2) Dataset Collection, 3) Expert Identification, and 4) Sur vey Application. Figure 1 presents the steps of our research, which are discussed next. For Library Selection (Section 2.3), we selected the top10 most popular libraries in the Java pro gramming language to identify library experts. We also se lected 6 libraries for microservices to favor external validity. In the Dataset Collection step (Section 2.3), we clone the projects that contain these libraries from GitHub. For Iden tification of Library Experts (Section 3.1), we compute the skills of developers based on three metrics: Number of Com mits, Number of Imports, and Lines of Code. These metrics are presented in Section 3.1. Finally, we performed two sur vey studies. These surveys were conducted to evaluate the precision of the strategy according to the responses of devel opers. Section 4.1 and 5.1 present details about the surveys.

Dataset
To create our dataset, we select the 10 most popular and com mon Java libraries among GitHub developers: Hibernate, Se lenium, Hadoop, Spark, Struts, GWT, Vaadin, Primefaces, Apache Wicket, and JavaServer Faces. This selection was made based on a survey provided by Stack Overflow 1 in 2018 with answers of over 100,000 developers around the world. Table 2 summarizes the definitions of each library (top10). All definitions of the libraries were retrieved from Stack Overflow and their Web pages. We selected Java be cause it is one of the most popular programming languages 2 and there are many Java projects available on GitHub.
Microservices have become most popular in the last years, together with the spread of DevOps practices (Pahl, 2015). We can see a significant increase in the use of microservices architectural style since 2014 (Klock et al., 2017), which can be verified in the serviceoriented software industry where the usage of microservices has been far superior when com pared to other software architecture models (Alshuqayran et al., 2016). Furthermore, a microservice usually runs on its own process and communicates using standardized inter faces. In practice, microservices are widely used by large Web companies, such as Netflix and Amazon (Alshuqayran et al., 2016). For these reasons, we aim to identify experts of microservices in 6 libraries: Apache Karaf, Apache Spark, JavaEE, Netflix, Spring Boot, and Swagger. Table 3 also summarizes the definitions of each library, but now con cerning microservices. These definitions were retrieved from Stack Overflow and their Web pages. Figure 2 illustrates the criteria for defining our dataset. To achieve more realistic results for software development, we apply the following exclusion criteria. (1) We excluded sys tems with less than 1 KLOC because we considered them toy examples or earlystage software projects. (2) We removed projects with no commit in the last 3 years because the devel opers may forget their code (Krüger et al., 2018). Finally, in the last exclusion criteria, (3) we removed projects which did not contain imports related to the selected libraries. Besides, we excluded all official projects of these libraries because we assume all library project developers are experts in the corresponding library. In popular Java libraries, we also re moved libraries with less than 100 projects (e.g., JavaServer Faces). We need a representative number of projects to eval uate our strategy. We analyze only files with extension .java. The same process was made to projects of libraries of mi croservices. Therefore, we end up analyzing 15 libraries in this study.  Table 4 shows the number of remained projects after each step in our filtering process. The first part of this table shows the results for top10 Java libraries, and the second part 1 https://insights.stackoverflow.com/survey/2018#mostpopular technologies 2 https://spectrum.ieee.org/static/interactivethetopprogramming languages2018 Hadoop A library that facilitates the use of the network from many computers to solve problems involv ing massive amounts of data (Tong et al., 2016; Ye, 2017.
Spark A generalpurpose distributed computing engine used for processing and analyzing a large amount of data Struts It helps in developing Webbased applications.
GWT It allows Web developers to develop and maintain complex JavaScript frontend applications in Java.
Vaadin It includes a set of Web components, a Java Web library, and a set of tools and application starters. It also allows the implementation of HTML5 web user interfaces using the Java.
PrimeFaces A library for JavaServer Faces featuring over 100 components.
Apache Wicket A library for creating reusable components and of fers an objectoriented methodology to Web devel opment while requiring only Java and HTML.
JavaServer Faces A Java view library running on the server machine which allows you to write template text in client side languages (like HTML, CSS, JavaScript, etc.).
shows the results for microservices libraries. The column #Projects presents the number of projects initially selected.
Next, the column Filtered shows the number of projects re moved through the filtering step. Finally, the column Re mained presents the number of projects analyzed for each library.

Applicability Evaluation
In this section, we describe how we evaluated the strategy in terms of its applicability focusing on the top9 Java libraries. Section 3.1 presents the steps to identify library experts, for example, metrics and data about classes. Section 3.2 shows an overview of our data. Section 3.3 presents the top10 ex perts in each library selected in this study.

Identification of Library Experts
To evaluate the strategy in terms of its applicability, we perform three steps in this study. These three steps are described as follows.
Step 1: Extract data from source code -In this step, we obtain data from the classes created by developers from a Git repository. All data, such as added or removed LOC, written imports, commits, date, email, and developers' names, are stored locally.

Library Description
JavaEE The JavaEE platform is built on top of the Java SE plat form. The Java EE platform provides an API and run time environment for developing microservices and run ning largescale, multitiered, scalable, reliable, and se cure network applications. Spring Boot Pivotal solution for implementing cloudbased microser vices using the well known Spring Framework. Netflix Netflix OSS is a set of frameworks and libraries that Netflix wrote to implement microservices in distributed systems. Swagger Swagger is used to creating documentation for each mi croservice. Karaf Apache project referenced to support microservice im plementations. Spark A lightweight web framework that has been used to im plement simple and expressive microservices. Step 2: Search for imports -From the previous step, we search for specific "imports" related to the chosen library. The idea is to explore all files that import the name of the target library. This step is performed as follows. First, the strategy gets files with all commits, for example, commits to LOC in general, comments, and mainly the header. Second, it analyzes the header of Java files containing the name of the package, all imports necessary to class, and the classes' names. Consequently, we get the "import" through regular expression pattern import+"target library", for example, "import org.apache.spark". In this example, the target library is Spark. Figure 3 shows an example of a file with data of committers. As we can observe in Figure 3, there are three attributes in this file: (1) hash code of commit, (2) name of the developer, and (3) committed source code. At the beginning of the file, there is the name of a package and many imports. In this part, our strategy is to use a regular expression to detect if the line contains the library we investigate. If the line contains the target library, we compute the hash of commit, the number of imports to the specific library, and the total imports without relation to the target library.
Step 3: Calculate skills -In this last step, we compute the skills for each developer. We rely on three metrics to identify library experts. Each metric is calculated concerning the number of commits to a specific library. That is, when a commit using a library is identified, the metrics were calculated. In the following, we explain the 3 proposed metrics.

Number of Commits.
This metric calculates the activity of each developer through the number of commits using a particular library. Through this metric, we believe it is possible to measure the library's amount in a project that a specific developer works.

Number of Imports.
This metric presents the extension of knowledge in the library. For this metric, we count all im ports to the library written by a developer. Repeated imports are included. If a developer wrote two equals imports, we would consider 2 imports to the target library. Figure 3 shows an example of repeated imports. There are four imports to Apache Hadoop in this figure, so we compute 4 imports for this library. Besides, if a developer made 3 imports to the same library, we compute 3 imports, for example, we are sup posed to developer made 3 imports. Lines of Code. To compute this metric, we developed a heuristic to count the amount of LOC related to a specific library. First, we obtain the ratio of changed LOC by the number of all imports in the file. Then, we multiply the ratio by the number of imports related to the library. Our heuris tic considers 3 attributes, the number of library imports, the number of imports in general, and the number of LOC al tered by a commit related to the library. The heuristic is then computed as follows: LOC = # of LOC Altered by Commit # of All Imports X # of Library Imports From Figure 3, it is possible to compute an ex ample for this metric. A developer made a commit with hash code 75b70c and an import to "import org.apache.hadoop.io.IntWritable;" (line 2). Therefore, we compute this metric as presented above and consider 10.67 LOC related to the Hadoop library.

Overview of Dataset
From the dataset projects, we computed all commits with the libraries evaluated in this study and identified 1.5 million dif ferent developers who made commits. Figure 4 shows the number of developers for the top9 popular Java libraries. The library, with more developers that made commits, was Selenium. This library has 811,884 developers. In contrast, Apache Wicket was the library with fewer developers: 5,440. It is important to say that these developers made at least one commit for the respective library. However, we cannot con sider them all experts since a single library use may not indi cate high expertise.  Figure 5 presents the results to the Number of Commits per library. Figure 6 presents an overview of the metric Number of Im ports per library. Finally, Figure 7 shows the results of the metric Lines of Code per library. In general, LOC (Figure 7) was the metric that presented more variation in our data set. For instance, GWT has developers that wrote more than 130 KLOC. Similarly, for Hibernate, it is possible to see an outlier developer who wrote more than 500 KLOC. In con trast, some developers wrote less than 10 lines of code, for example, to the library PrimeFaces.

Top Library Experts Selection
In this section, we present the Applicability Evaluation re sults to verify the feasibility of library expert identification focusing on the top9 popular Java libraries. We analyzed 16,703 software systems mined from GitHub and 9 libraries: Hibernate, Selenium, Hadoop, Spark, Struts, GWT, Vaadin, Primefaces, and Apache Wicket. Besides, we analyzed data from more than 1.5 million developers who have contributed to these projects in our dataset. Table 5 presents the results of top library experts. To ob tain these results, we aim to select the top10 developers, but, in some cases, it was not possible to select top10 developers. For instance, we obtained 3top developers in library Spark. Besides, we consider a developer with a library expert only if this developer obtains high values in at least two metrics, for example, LOC & # of commit or # of imports & LOC. These developers are identified concerning their contribution. For this, we calculate the 90% percentile in each metric, then fil tering the developers with any metric below this threshold (90%). This type of classification is common in other stud ies (Joblin et al., 2017; Ferreira et al., 2019. Finally, we sort developers by LOC (# of Library LOC). The filtering thresh old was applied to remove potential false positives (i.e., de velopers with high # of Library LOC, but low # of Commits). In some cases, it resulted in less than 10 experts for some libraries, such as PrimeFaces (8), Spark (3), Struts (6), and Wicket (5).
In Table 5, each developer is identified by the start name of the library, followed by a sequence number (e.g., HAD (1) means the first developer expert of Hadoop). The column # of Library Imports refers to the metric of Number of Imports written by the developer. It counts the number of imports re lated to the specific library evaluated in this study. The col umn # of All Imports shows the number of imports wrote by the developer in general. When a developer wrote an import to a specific library evaluated in this study, they also wrote imports to other libraries that have not been evaluated. Hence, this metric counts all imports in relevant commits made by the developer.
The column # of Commits shows the results for the Num ber of Commits metric. This metric indicates the number of commits made by a specific developer. The column # of LOC On the other hand, we classify a developer as Soft using the same strategy to classify the Hard developers. However, we use the data below 25%, i.e., the 1 st quartile, as a parameter. Then, we discuss the below reasoning regarding this classification.

Hard Committers and Hard Coders.
According to our metrics, the developer GWT (3) is a Hard Committer and Hard Coder (see Table 5). This developer made more than 450 commits and wrote more than 55 KLOC for this library. It could be noted that other developers are Harder Committer and Harder Coder. For instance, the developer HAD (1) made 172 commits and wrote more than 237 KLOC. These are some examples of Harder Committer and Harder Coder from the calculated metrics.

Hard Committers and Soft Coders.
We present now the results to Hard Committers and Soft Coders. Developers HAD (1) and HAD (4) in Table 5 can be considered Hard Committers because they made 172 and 146 commits, respectively. The difference between HAD (1) and HAD (4) is only 26 commits. However, developer HAD (4) is considered as Soft Coder concerning developer HAD (1) because HAD (1) wrote more than 235 KLOC while the developer HAD(4) wrote about 15 KLOC. Developer HAD (4) wrote only 6% LOC of the developer HAD (1). Therefore, HAD (4) is a Hard Committer and Soft Coder.

Soft Committers and Hard Coders.
Concerning the Soft Committers and Hard Coders, we can observe that developers PRI (1), PRI (2), SEL (1), and STR (1) in Table 5 are Soft Committers because they made only a few commits. Developer STR (1), for instance, made only 3 commits, but s/he wrote more than 13 KLOC. Therefore, this developer is considered a Soft Committer and Hard Coder.

Soft Committers and Soft Coders.
As the name suggests, this category includes the developers that fewer commits and made fewer lines of code compared to their peers. For in stance, Developers HIB (9), HIB (10), SEL (9), and SEL (10) are considered Soft Committers because they wrote less than 20 commits to libraries cited. Besides, these developers wrote less than 5 KLOC. Therefore, according to our met rics, these developers are considered Soft Committers and Soft Coders.

Survey with Top Libraries Experts
This section describes the survey applied to GitHub develop ers to evaluate the strategy with respect to the top9 popular Java libraries. Section 4.1 presents the details regarding the survey developed. Section 4.2 presents a summary of some relevant findings. Section 4.3 presents the results to RQ1 re garding the Number of Commits metric. Section 4.4 presents the results to RQ2 about the Number of Imports metric. Sec tion 4.5 presents the results to RQ3 regarding the LOC met ric.

Survey Design
According to Easterbrook et al. (2008), survey studies are used to identify the characteristics of a population and are usually associated with the application of questionnaires. Be sides, surveys are meant to collect data to describe and com pare or explain knowledge (Pfleeger and Kitchenham, 2001). We selected the library experts with the best values in the evaluated metrics to validate them through a survey. We de signed and applied a survey with the top developers identi fied by our strategy. We selected developers with the top 20% highest values in at least two (out of three) metrics.
We created a questionnaire on Google Forms 3 with two parts: the first one was composed of 5 questions about the background of the expert candidates; the second part also had 5 questions about the knowledge of the expert candi dates regarding the evaluated libraries. Table 6 contains the tag <libray name> meaning a specific library, for instance, Hadoop. Also, this table shows the possible answers to the survey questions. To obtain the email used by the developer to perform the commits in the source code, we used the GitBlame 4 tool. The emails were collected to send the survey. We sent an email to developers asking them to assess their knowledge of each library. For instance, the developers were invited to rank their knowledge (Table 6, SQ1) using a scale from 1 (one) to 5 (five), where (1) means no knowledge about the library; and (5) means extensive knowledge about the library. Ques tions are not mandatory because they may require knowledge of the exceptional features of the library. Therefore, partici pants are not forced to provide an answer when they do not re member a specific library element, such as the time of devel opment using the library and the approximate frequency of commits that contain the library. The survey remained open for 15 days in January 2019.
In summary, we present the precision evaluation results based on a survey with expert candidates in each of the top9 popular Java libraries. The goal of this evaluation is to verify the precision of the library expert identification. We empir ically selected 1,045 developers among the top20% values in at least 2 metrics. The questionnaire was sent in January 2019. After 15 days, we obtained 137 responses resulting in a response rate of about 15%. We asked the 137 develop ers about their software development experience in general (background) and the use of the specific libraries investigated in this paper.

Overview
In this section, we present an overview of some relevant find ings of the popular Java libraries. Table 7 presents an overview of the experts' candidates contacted to answer our first survey. This table has the fol lowing structure. The first column (Library) indicates the name of the analyzed library. The second column (Emails sent) shows the number of emails collected and sent to ex pert candidates. The third column (Invalid email) presents the number of invalid emails returned by the server. The fourth column (Remaining emails) indicates the number of valid emails. The fifth column shows the number of answers we obtained for each library. Finally, in the last column, we show the response rate of each library. Concerning the participants' background and replication package, we create a Web page with more details (Oliveira et al., 2020). It is worth mentioning that half of the respon dents graduated in Computer Science, and 7% holds a Ph. D. degree. Concerning time dedicated to software development, 47% has more than 10 years of experience, and only 2% have less than 1 year of experience. Therefore, we can conclude that, in general, the participants are not novices.
Our study also shows that a significant amount of expert candidates makes commits. When writing code related to a specific library, they perform many imports of particular li braries and writes lines of code about the library. We support this affirmation through metrics that evaluate the amount of LOC written by a developer when they performed a commit. Table 8 shows the results of the knowledge that surveyed de velopers claim to have in each library. If we analyze the data about the precision of the strategy from the sum of levels 3, 4, and 5 of the Likerttype scale, we obtain on average 88.49% of precision about the knowledge of the developers, i.e., identification is correct in more than 88% of the cases. On the other hand, although a score of three may represent acceptable knowledge, if we followed more conservative cri teria, only classifying as library experts the developers that informed a higher (≥ 4) knowledge on the libraries obtain average, 63.31% of precision. This way, we conclude that less than 2/3 of the identified expert candidates identified by the strategy contain high knowledge about the evaluated li braries.
About 63% of the library experts who answered the survey have high knowledge about the evaluated libraries.

Level of Activity
In this section, we answer the first research question.

RQ1-How to evaluate the level of activity of a developer in a library?
To answer this research question, we asked the library ex perts the following question. ''How often are your commits related to the <libray name> library''? Figure 8 shows the results of this question in the first line in each chart to each library. For most libraries, the majority of the participants answered they made ''few'' commits using the evaluated li braries. This way, if we evaluated the results obtained for this label, it is possible to see that from 137 experts, 54% made ''few'' commits. For instance, in the library Hibernate, 87% of developers said they made few commits related to this library. Another library that deserves special attention is Struts. In this library, 88% of the developers responded that they made few commits. Regarding the label ''a lot'', only 39% of experts polled said they performed many commits. GWT was the library with a higher rate of answers to this label (62%). Therefore, results indicate that the metric Num ber of Commits needs to be combined with other metrics to achieved conclusive results about the skill of developers and even develop other metrics to identify the level of activity ability.
Answer to RQ1. A large proportion of library experts make ''few'' commits using the library. Therefore, we con cluded that the solo use of the number of commits could not identify library experts.

Knowledge Intensity
In this section, we answer the second research question.

RQ2-How to evaluate the knowledge intensity of a devel oper in a library?
Regarding the number of imports to indicate a library ex pert, we ask the developers the following question: ''How often do you include an import of <libray name> library in your commits?''. Figure 8 shows the results of this question from the second line in each chart to each library. We ana lyze the number of imports performed by developers. The main reason for this analysis is to evaluate the feasibility of inferring the skills of the developers from the types of written imports. In general, the label ''few'' and ''a lot'' are tied or with little difference between them. For example, Hibernate, Spark, and PrimeFaces have practically tied. These libraries did not show significant differences; the difference was only 1 absolute point in some cases. In only three cases, the label ''a lot'' remained significantly higher: GWT (83%), Vaadin (67%), and Selenium (78%).
From 137 experts, 68% said that they made ''a lot of im ports''. However, the number informed by the experts indi cates that this metric requires a combination with other met rics to achieve better results because 32% of experts said they made few imports to libraries evaluated. Therefore, from the survey results, the metric Number of Imports, as well as the metric Number of Commits, are not able to identify library experts when we apply one at a time.
Answer to RQ2. The metric Number of Imports is not able to identify library experts, when we use it alone.

Knowledge Extension
In order to evaluate the metric Lines of Code, we present the third research question as follows.

RQ3-How to evaluate the knowledge extension of a devel oper in a library?
In this research question, we analyze the developers skill from the number of LOC related to the library. We evaluate the number of LOC implemented by a developer to a specific library. For this purpose, we asked the library experts from the survey the following question. ''How much of your code is related to the <libray name> library when you perform a commit?''. Figure 8 shows the results to this question in the third line in each chart to each library. The libraries GWT, Wicket, Selenium, and Hadoop, for instance, obtained 74%, 71%, 70%, and 64% respectively to label ''a lot''.
We noted, however, the label ''a few'' also remained at a high level in some cases, for instance, the libraries Struts (88%) and Spark (55%). In fact, the library Hibernate re mained tied to labels ''a few'' and ''a lot''. In general, from 137 experts, 39% said they write ''a few'' LOC and 61% write ''a lot'' LOC with respect to libraries. Therefore, it is possible to infer that the metric Lines of Code alone also does not provide indications about developer skills, although this metric achieved better precision than the metric Number of Commits.
Answer to RQ3. According to our analysis, the metric Lines of Code alone cannot reliably provide indications about developers' skills. In general, our metrics are not fea sible to identify library experts. However, our strategy is able to reduce the search space of library experts. There fore, a company or project open source can be select a de veloper from a group selected by our strategy.

Survey with Microservices Experts
In order to favor the generalization of our findings, we did a second survey with developers of microservices libraries. For this, we conducted a selection of libraries in this domain.

Survey Design
We select the library experts to this survey in a similar way to the survey presented in Section 4.1. We created a ques tionnaire on Google Forms 5 in order to evaluate the knowl edge of the developers about microservices libraries. The first question request the login of the developer at GitHub. This login is necessary to map the answer of the developer with our data. We ask developers about their knowledge in all six libraries of microservices. For this, we show all li braries investigates in this survey (6 libraries of microser vices). We request the developer to rank their knowledge in these libraries in four levels: No knowledge, Low knowledge, Medium knowledge, and Extensive knowledge. Each level of knowledge has meaning. No knowledge: this library was never used in any project I am involved in. Low knowledge: I never used this library, but it has been used in projects I am involved in. Medium knowledge: I used this library in some projects before, but I do not master all its API. Exten sive knowledge: I used this library many times, and I know a lot of its API. Table 9 shows the template of the survey.
We selected the library experts with the best values in the evaluated metrics to validate them through a survey. We de signed and applied the survey with the top developers iden  Figure 8. Results of the survey questions for each library tified by our strategy. That is, we selected developers with the top20% highest values in at least two (out of three) met rics. Therefore, we choose 136 candidates library experts in microservices. Figure 9 presents an overview of the number of developers by the library. The library with more candidate experts identified was Netflix with 64, and the library with fewer candidate library experts identified was Karaf with only 1.  Table 10 presents an overview of the candidate experts contacted to answer our survey. We sent 136 emails, but 38 was returned with invalid email. Therefore, we sent the sur vey to 98 valid emails. The library, with more amount of respondents, was Netflix with 7 candidate library experts. On the other hand, the library with fewer participants was Apache Karaf with 0.

Results
In this section, we present the results about a survey per formed with library experts from microservices. Initially, we perform a pilot survey with 5 developers from Netflix ran domly selected among the candidate experts identified. We received 3 answers for this library. From the pilot, we did not identify any problem with our survey. Then we apply the final survey for all top20% developers with high values in at least two metrics. Note that the results of the pilot survey are part of our final results. Table 11 shows the summary results from our second sur vey. The first column shows the name of the library. The sec ond column shows the number of developers without knowl edge in the library target. The third column indicates the number of developers with low knowledge in the library tar get. The fourth column shows the number of developers with medium knowledge in the library target. The fifth column shows the number of developers with extensive knowledge in the library target. Finally, the two last columns show the precision for medium and extensive knowledge. The library Apache Karaf is not presented in Table 11 because we did not obtain any response for this library.  Table 12 presents the overview of the survey applied with developers from microservices. This table has 8 columns. The column Developer represents the name of the developer. We omitted the name of developers to avoid his/her expo sure. Next, six columns in the sequence represent the libraries investigates from the survey. Finally, we have the target li brary target name. This column represents the library that our strategy classified the developer as library experts. In this table, we have 4 scales for developers to rank her/his knowledge. The NK represents ''no knowledge'', LK rep resents ''low knowledge'', MK indicates ''medium knowl edge'', and EK represents ''extensive knowledge''. Table 12 shows, for instance, developerD1 was identified by our strat egy as medium knowledge or extensive knowledge from li brary Spark. However, developer D1 answered that he/she has low knowledge in this library. D1 is an interesting case because this developer reports low knowledge in the library they had been recommended, but medium Knowledge and extensive knowledge for all others. Netflix is also an inter esting case since only 3 (out of 8) reported extensive knowl edge in the library, while 5 reported extensive knowledge in JavaEE and 5 in Spring Boot. On the other hand, developer D17 was identified by our strategy as medium knowledge or extensive knowledge from library Spark, and this developer marked extensive knowledge.
From 21 developers that answered the survey, we observe that the strategy obtained a precision of 52% on average for extensive knowledge. Table 11 in the last column shows, for example, to library SpringBoot a precision of 83%. On the other hand, for library Netflix, our strategy obtained a pre cision of 43%. If we consider the survey results to corre late with the results of the strategy concerning only the de velopers that answered with extensive knowledge, we ob tained 52% precision. However, if we consider developers who answered the survey with medium knowledge or exten sive knowledge to correlate with strategy results, we obtain 71% precision.

Tool Support
We developed a prototype tool, named JExpert, to support the identification of library experts (strategy). We developed JExpert in Java programming language. JExpert currently works with Java projects, but the tool can be easily adapted to identify library experts in other programming languages. JExpert is a standalone tool and runs in Windows, Linux, and MAC. JExpert is available in our website (Oliveira et al., 2020). JExpert uses static analysis to avoid Abstract Syntax Tree (AST). Therefore, it reduces the response time when analyzing large systems with hundreds of source elements, such as LOC, imports, packages, and classes. Our goal is to support recruiters with a flexible, lightweighted means to identify library experts from source code. Figure 10 presents the simplified architecture design of JExpert. In the first moment, there are two modules: Projects and Library Name. These two modules are the input of JEx pert. In other words, JExpert receives as inputs two items, (i) projects in Java that contain the target libraries, i.e., sys tems from a local directory informed by the user, and (ii) the names (keywords) of the libraries that a developer wants to investigate. Module Activity Extractor is responsible for ex tracting the code elements necessary for the computation of activities made by a developer. Besides, this module removes the old projects, i.e., projects with commits made more than three years ago, projects with less than 1 KLOC, and projects without target library. From the next step, the module Developer Data Analyzer computes all data about each developer. This module is re sponsible for separating the number of commits to libraries and changes made from source code in general, for instance, the number of lines of code written. This module also com putes the number of imports made by developers and verifies if an import is related to the target library.
The Metric Collector module computes the three metrics, as mentioned in Section 3.1. Finally, the List of Experts is generated as output with the sorted list of expert candidates from our metrics. Such a list prioritizes the library experts based on a heuristic score, i.e., higher scores come first; cur rently, the tool returns a ".csv" file for each library.

Threats to Validity
We based our study on related work to support the evalu ation of a strategy to identify library experts. Regarding the assessment, we conducted a careful empirical study to assess the efficiency of the strategy from software systems hosted by GitHub. The strategy evaluated can analyze source code from platforms that follow the Git architecture. However, some threats to validity may affect our research findings. The main threats and respective treatments are discussed below based on the proposed categories of Wohlin et al. (Wohlin et al., 2012).

Construct Validity.
This validity is related to whether mea surements in the study reflect realworld situations (Wohlin et al., 2012). Before running the strategy, we conducted careful filtering of software systems from GitHub reposito ries. However, some threats may affect the correct filtering of systems, such as human factors that wrongly lead to a valid system's discard to be evaluated. Considering that the exclusion criteria to system selection were applied in a manual process, we may have discarded interesting systems that we identified as nonJava, for instance.
Internal Validity. The validity is related to uncontrolled aspects that may affect the strategy results (Wohlin et al., 2012). The strategy may be affected by some threats. To treat this possible problem, we selected a sample of 5 software systems that contain the library Hadoop from our dataset, with a diversified number of LOC. Then, we manually identified the number of commits from the GitHub repository, the number of imports, and the number of LOC codified to the specific library. We compared our manual results with the results provided by the tool and observed a loss of 5% in metrics terms computed through the automated process. We believe that this error rate does not invalidate our main conclusions. In addition, our strategy has the goal to reduce the search space to identify library experts, that is, we do not recommend a specific developer.
External Validity. This validity is related to the possibility of generalizing our results (Wohlin et al., 2012). We evalu ated the strategy with a set of 16,703 software projects from GitHub. Considering that these systems may not include all existing libraries, our findings may not be generalized. Fur thermore, we evaluated the strategy with an online survey with only 158 developers that implemented projects with the investigated libraries. We analyzed the data with only 15 Java libraries. However, we chose the top libraries from the survey reported by StackOverflow in 2018, with over 100,000 re sponses from developers around the world. We also analyzed microservices libraries. This way, we believe these libraries can represent a reasonable option to evaluate the strategy.

Related Work
The use of data from GitHub to understand how software de velopers work and collaborate has become recurrent in soft ware engineering studies (Greene and Fischer, 2016; Singer et al., 2013; Ortu et al., 2015; Destefanis et al., 2016; Ma et al., 2009; Begel et al., 2010; Moraes et al., 2010. Some studies seek to understand the behavior of developers con cerning an interaction with their peers (Ortu et al., 2015). For example, a few studies (Ortu et al., 2015 tried to understand who are the developers with peaceful behavior and those with aggressive behavior and if these developers coexist productively in software development projects . Similar studies also tried to understand if there is a relationship between bug resolution time and behavior of developers (Ortu et al., 2015). Also, some studies investi gated developers manners  and seek to understand the emotional behavior of software develop ers . Schuler and Zimmermann (2008) investigated developer expertise based on their commit activities, which manifests itself whenever developers are using functionality. They present preliminary results for the Eclipse project. They were able to create expertise profiles that included data about what APIs a developer may be an expert in through their use of those APIs. Wu et al. (2011) proposed DREX, an approach to bug assignment using knearest neighbor search and social network analysis. This approach performs with the follow ing way: 1) finding textually similar bug reports, 2) extract ing developers involved in their resolution, and 3) ranking the developers expertise by analyzing their participation in resolving similar bugs. An evaluation of bug reports from the Firefox OSS project shows the social network analysis of DREX outperforms a purely textual approach, with a pre diction accuracy of about 15%.
In closely related work, Greene and Fischer (2016) have developed an approach to extract technical information from GitHub developers. The work of these researchers does not differentiate developers from their level of knowledge of technical skills since a recruiter has several candidates for the same job position. Besides, such work only shows the profile of the users in GitHub, and it does not extract other charac teristics of their knowledge and skills. The other limitation is that they neither provide actual data about the developer's knowledge production nor present a survey to evaluate the results. Singer et al. (2013) investigated the use of profile ag gregators in the evaluation of developer skills by developers and recruiters. However, these aggregators only gather skills for individual developers, and it is not clear how they support the identification of relevant developers from a large dataset. We believe that the strategy evaluated in our study is com plementary to the described related work, providing a differ ent approach focusing on reducing the search space to iden tify possible experts. Our strategy is complementary with other approaches, such as CVExplorer (Greene and Fischer, 2016). For instance, by combining our results with CVEx plorer (Greene and Fischer, 2016) it is possible to select skills in the language of programming and analyze the met rics shown in our paper. To the best of our effort, we did not find a similar large scale study that evaluates some strategy able to identify library experts. Hence, we cannot compare the strategy evaluated with other studies.

Conclusion
In this paper, we evaluated a strategy to reduce the search space to identify library experts in software systems from source code analysis. We also presented a prototype tool that implements the strategy. The strategy evaluated is composed of three metrics: Number of Commits, Number of Imports, and Lines of Code. We assessed the strategy in two dimen sions: applicability and precision. First, Applicability Evalu ation analyzed the feasibility of identifying library experts candidates in large datasets. Second, Precision Evaluation compared the results provided by a strategy with developers perceptions from a survey. In total, we analyzed 16k software systems mined from GitHub, 15 libraries, and a survey with 158 developers. Our findings pointed out that the strategy was able to identify library experts in different libraries from the set of input software systems with a precision of 71% on average.
There are many possible extensions for this work. For in stance, we did not consider all available data in our analysis, such as the number of forks, number of projects belonging to the developer that have received stars, the number of follow ers, number of methods, source code quality, and contribu tions to the project discussions. Besides, we did not consider the number of lines of code added and removed between ver sions. Future work can also extend our research to evaluate the strategy of other programming languages and libraries.