Large Language Model-based suggestion of objective functions for search-based Product Line Architecture design
Resumo
Search-based design of Product Line Architecture (PLA) focuses on enhancing the design and functionality of software product lines through variability management, reuse, and optimization. A particular challenge in this area is the selection of objective functions, which significantly influence the success of the search process. Moreover, many objectives make the analysis and choice of a solution to be used harder. The literature has assigned this task to the PLA designer, i.e., the Decision-Maker (DM), who does not always know all the functions and their impact on the optimization outcomes. On the other hand, recent research shows that Large Language Models (LLMs), particularly the Generative Pre-trained Transformer series (GPT), have obtained promising results to help in various Software Engineering (SE) tasks. Considering this fact, this work explores the integration of such LLMs, notably ChatGPT, into the search-based PLA design. By leveraging LLMs’ capacity to understand/generate human-like text, we investigate their potential to assist DMs and propose an approach for suggesting objective functions, thereby simplifying and improving decision-making in PLA design optimization. Through empirical tests and qualitative feedback from domain experts, this research highlights the application of LLMs in search-based SE. The results demonstrate that integrating ChatGPT into PLA design can significantly enhance decision-making efficiency and solution quality, with a 40% reduction in time required for selecting objective functions and a 25% improvement in solution quality from the DM’s point of view. This study maps out the challenges and opportunities that lie ahead in fully harnessing their potential for PLA search-based design.
Palavras-chave:
Product Line Architecture, Objective Function Selection, Large Language Model
Referências
Mathieu Acher and Jabier Martinez. 2023. Generative AI for Reengineering Variants into Software Product Lines: An Experience Report. In Proceedings of the 27th ACM International Systems and Software Product Line Conference-Volume B. 57–66.
David Benavides, Sergio Segura, and Antonio Ruiz-Cortés. 2010. Automated analysis of feature models 20 years later: A literature review. Information Systems 35, 6 (2010), 615–636.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
Paul Clements and Linda Northrop. 2002. Software Product Lines: Practices and Patterns. Addison-Wesley Professional.
Thelma Elita Colanzi, Silvia Regina Vergilio, Itana Gimenes, and Willian Nalepa Oizumi. 2014. A search-based approach for software product line design. In Proceedings of the 18th International Software Product Line Conference (SPLC), Vol. 1. 237–241. DOI: 10.1145/2648511.2648537
Kalyanmoy Deb and Himanshu Jain. 2014. An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints. IEEE Transactions on Evolutionary Computation 18, 4 (Aug 2014), 577–601.
Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and Tamt Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (2002), 182–197.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1. 4171–4186.
Paula M. Donegan and Paulo C. Masiero. 2007. Design Issues in a Componentbased Software Product Line.. In SBCARS. Citeseer, 3–16.
Willian M. Freire, Murilo Boccardo, Daniel Nouchi, Aline M. M. M. Amaral, Silvia R. Vergilio, Thiago Ferreira, and Thelma E. Colanzi. 2024. Complementary Material. DOI: 10.6084/m9.figshare.25556157
Willian Marques Freire, Mamoru Massago, Arthur Cattaneo Zavadski, Aline Maria Malachini Miotto Amaral, and Thelma Elita Colanzi. 2020. OPLA-Tool v2. 0: a Tool for Product Line Architecture Design Optimization. In 34th Brazilian Symposium on Software Engineering (SBES). Association for Computing Machinery, New York, NY, USA, 818–823.
Mark Harman, S. Afshin Mansouri, and Yuanyuan Zhang. 2012. Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys (CSUR) 45, 1 (2012), 11.
Frank van der Linden, Klaus Schmid, and Rommes Eelco. 2007. The product line engineering approach. In Software Product Lines in Action. Springer, 3–20. DOI: 10.1007/978-3-540-71437-8_1
Mohamed Wiem Mkaouer, Marouane Kessentini, Slim Bechikh, Kalyanmoy Deb, and Mel Ó Cinnéide. 2014. High dimensional search-based software engineering: Finding tradeoffs among 15 objectives for automating software refactoring using NSGA-III. In Proceedings of the 16th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO’14). ACM, Vancouver, Canada, 1263–1270.
Luiz Fernando Okada. 2023. Guidelines to Support OPLA-Tool Adoption for Novice Users. Undergraduate Monograph. State University of Maringá, Maringá-PR, Brazil. Advisor(s) Thelma Elita Colanzi Lopes.
Ipek Ozkaya. 2023. Application of large language models to software engineering tasks: Opportunities, risks, and implications. IEEE Software 40, 3 (2023), 4–8.
Klaus Pohl, Günter Böckle, and van Der L Frank J. 2005. Software product line engineering: foundations, principles and techniques (1 ed.). Springer Science & Business Media.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. (2018).
Alec Radford, JeffreyWu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019). [link]
Marcelo C. B. Santos, Thelma E. Colanzi, Aline M. M. M. Amaral, and Edson OliveiraJr. 2017. Preliminary Study on the Correlation of Objective Functions to Optimize Product-Line Architectures. In Proceedings of Brazilian Symposium on Software Components, Architectures, and Reuse. SBCARS’17, Fortaleza-CE, Brazil, 10.
SEI. 2009. Software Engineering Institute - The Arcade Game Maker Pedagogical Product Line. [link]. Accessed in 2018 August.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. Advances in Neural Information Processing Systems 30 (2017).
Yenisei Delgado Verdecia, Thelma Elita Colanzi, Silvia Regina Vergilio, and Marcelo C. Santos. 2017. An Enhanced Evaluation Model for Search-based Product Line Architecture Design. In 20th Ib. Conference on Software Engineering (CIbSE). CIbSE, San Jose, Costa Rica, 155–168.
Xiaofei Wang, David Lo, Xin Xia, Shuai Li, and Jianling Sun. 2021. A Survey on Natural Language Processing for Software Engineering. IEEE Transactions on Software Engineering (2021).
Lucas Hideki Yamanaka. 2023. Study on the correlation of objective functions to optimize software product line architecture. Journal of Software Engineering Research (2023). In Portuguese.
David Benavides, Sergio Segura, and Antonio Ruiz-Cortés. 2010. Automated analysis of feature models 20 years later: A literature review. Information Systems 35, 6 (2010), 615–636.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
Paul Clements and Linda Northrop. 2002. Software Product Lines: Practices and Patterns. Addison-Wesley Professional.
Thelma Elita Colanzi, Silvia Regina Vergilio, Itana Gimenes, and Willian Nalepa Oizumi. 2014. A search-based approach for software product line design. In Proceedings of the 18th International Software Product Line Conference (SPLC), Vol. 1. 237–241. DOI: 10.1145/2648511.2648537
Kalyanmoy Deb and Himanshu Jain. 2014. An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints. IEEE Transactions on Evolutionary Computation 18, 4 (Aug 2014), 577–601.
Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and Tamt Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (2002), 182–197.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1. 4171–4186.
Paula M. Donegan and Paulo C. Masiero. 2007. Design Issues in a Componentbased Software Product Line.. In SBCARS. Citeseer, 3–16.
Willian M. Freire, Murilo Boccardo, Daniel Nouchi, Aline M. M. M. Amaral, Silvia R. Vergilio, Thiago Ferreira, and Thelma E. Colanzi. 2024. Complementary Material. DOI: 10.6084/m9.figshare.25556157
Willian Marques Freire, Mamoru Massago, Arthur Cattaneo Zavadski, Aline Maria Malachini Miotto Amaral, and Thelma Elita Colanzi. 2020. OPLA-Tool v2. 0: a Tool for Product Line Architecture Design Optimization. In 34th Brazilian Symposium on Software Engineering (SBES). Association for Computing Machinery, New York, NY, USA, 818–823.
Mark Harman, S. Afshin Mansouri, and Yuanyuan Zhang. 2012. Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys (CSUR) 45, 1 (2012), 11.
Frank van der Linden, Klaus Schmid, and Rommes Eelco. 2007. The product line engineering approach. In Software Product Lines in Action. Springer, 3–20. DOI: 10.1007/978-3-540-71437-8_1
Mohamed Wiem Mkaouer, Marouane Kessentini, Slim Bechikh, Kalyanmoy Deb, and Mel Ó Cinnéide. 2014. High dimensional search-based software engineering: Finding tradeoffs among 15 objectives for automating software refactoring using NSGA-III. In Proceedings of the 16th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO’14). ACM, Vancouver, Canada, 1263–1270.
Luiz Fernando Okada. 2023. Guidelines to Support OPLA-Tool Adoption for Novice Users. Undergraduate Monograph. State University of Maringá, Maringá-PR, Brazil. Advisor(s) Thelma Elita Colanzi Lopes.
Ipek Ozkaya. 2023. Application of large language models to software engineering tasks: Opportunities, risks, and implications. IEEE Software 40, 3 (2023), 4–8.
Klaus Pohl, Günter Böckle, and van Der L Frank J. 2005. Software product line engineering: foundations, principles and techniques (1 ed.). Springer Science & Business Media.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. (2018).
Alec Radford, JeffreyWu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019). [link]
Marcelo C. B. Santos, Thelma E. Colanzi, Aline M. M. M. Amaral, and Edson OliveiraJr. 2017. Preliminary Study on the Correlation of Objective Functions to Optimize Product-Line Architectures. In Proceedings of Brazilian Symposium on Software Components, Architectures, and Reuse. SBCARS’17, Fortaleza-CE, Brazil, 10.
SEI. 2009. Software Engineering Institute - The Arcade Game Maker Pedagogical Product Line. [link]. Accessed in 2018 August.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. Advances in Neural Information Processing Systems 30 (2017).
Yenisei Delgado Verdecia, Thelma Elita Colanzi, Silvia Regina Vergilio, and Marcelo C. Santos. 2017. An Enhanced Evaluation Model for Search-based Product Line Architecture Design. In 20th Ib. Conference on Software Engineering (CIbSE). CIbSE, San Jose, Costa Rica, 155–168.
Xiaofei Wang, David Lo, Xin Xia, Shuai Li, and Jianling Sun. 2021. A Survey on Natural Language Processing for Software Engineering. IEEE Transactions on Software Engineering (2021).
Lucas Hideki Yamanaka. 2023. Study on the correlation of objective functions to optimize software product line architecture. Journal of Software Engineering Research (2023). In Portuguese.
Publicado
30/09/2024
Como Citar
FREIRE, Willian M.; BOCCARDO, Murilo; NOUCHI, Daniel; AMARAL, Aline M. M. M.; VERGILIO, Silvia R.; FERREIRA, Thiago; COLANZI, Thelma E..
Large Language Model-based suggestion of objective functions for search-based Product Line Architecture design. In: SIMPÓSIO BRASILEIRO DE COMPONENTES, ARQUITETURAS E REUTILIZAÇÃO DE SOFTWARE (SBCARS), 18. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 21-30.
DOI: https://doi.org/10.5753/sbcars.2024.3833.
